Image processing device and method

ABSTRACT

The present disclosure relates to an image processing device and method that enable improvement in processing efficiency in encoding or decoding. 
     A type E0 of an LCU  115  and its coefficient are determined on the encoder side. The coefficient of the type E0 was sent to the decoder side at the time of the LCU  111 , and has been stored in an EO buffer included in an adaptive offset filter on the decoder side. For the LCU  115 , therefore, a copy of the coefficient of the type E0 in the EO buffer is used on decoder side without the sending of the coefficient of the type E0. As described above, the parameters of the adaptive offset filter, which are transmitted in one batch at the beginning of a frame in the related art, are sent sequentially at the beginning of an LCU for each LCU. The present disclosure can be applied to, for example, an image processing device.

TECHNICAL FIELD

The present disclosure relates to image processing devices and methods,and more specifically to an image processing device and method thatenable improvement in processing efficiency in encoding or decoding.

BACKGROUND ART

In recent years, devices which handle image information in the digitalform and, in this case, utilize redundancy specific to the imageinformation to achieve compression and coding of images by using acoding scheme for compression based on an orthogonal transform such as adiscrete cosine transform and motion compensation in order toefficiently transmit and accumulate information are becoming widespread.Examples of this coding scheme include MPEG (Moving Picture ExpertsGroup) and H.264 and MPEG-4 Part 10 (Advanced Video Coding, hereinafterreferred to as H.264/AVC).

In addition, standardization of a coding scheme called HEVC (HighEfficiency Video Coding) is currently being undertaken by JCTVC (JointCollaboration Team-Video Coding), which is a joint standardizationorganization of the ITU-T and the ISO/IEC, for achieving more improvedcoding efficiency than H.264/AVC (see, for example, NPL 1).

In the present working draft of HEVC, a deblocking filter, an adaptiveloop filter, and an adaptive offset filter (Sample Adaptive Offset: SAO)are employed as in-loop filters.

In HEVC, parameters of the adaptive loop filter are collectively sent tothe decoder side in groups of one frame. In contrast, NPL 2 has proposedthat an adaptive loop filter process is performed in units of a largestcoding unit, or LCU.

CITATION LIST Non Patent Literature

-   NPL 1: Thomas Wiegand, Woo-Jin Han, Benjamin Bross, Jens-Rainer Ohm,    Gary J. Sullivan, “Working Draft 4 of High-Efficiency Video Coding”,    JCTVC-F803, Joint Collaborative Team on Video Coding (JCT-VC) of    ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 6th Meeting: Torino, IT,    14-22 Jul., 2011-   NPL 2: A. Fuldseth, Cisco Systems, G. bjontegaard, Cisco Systems,    “Improved ALF with low latency and reduced complexity”, JCTVC-G499,    Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3    and ISO/IEC JTC1/SC29/WG11 7th Meeting: Geneva, CH, 21-30 Nov., 2011

SUMMARY OF INVENTION Technical Problem

In HEVC, furthermore, the adaptive offset filter is adapted to regionscalled quad-tree regions, which are uniquely defined for the adaptiveoffset filter. In addition, the parameters of the adaptive offset filterare collectively defined in sao_param( ) in groups of one frame.

sao_param( ) is placed before the data (video information) in an encodedstream. On the encoder side, therefore, it is necessary to hold data forone frame in a buffer until the completion of the adaptive offset filterprocess, the determination of the coefficients of the adaptive offsetfilter, and the creation of sao_param( ).

The present disclosure has been made in view of such a situation, andenables improvement in processing efficiency in encoding or decoding.

Solution to Problem

An image processing device of a first aspect of the present disclosureincludes an acquisition unit that acquires parameters of an adaptiveoffset filter in units of a largest coding unit from an encoded streamin which the parameters of the adaptive offset filter are set using thelargest coding unit as a unit of transmission; a decoding unit thatperforms a decoding process on the encoded stream and that generates animage; and an adaptive offset filter unit that performs adaptive offsetfiltering on the image generated by the decoding unit, in units of alargest coding unit using the parameters acquired by the acquisitionunit.

The parameters of the adaptive offset filter can include a type of theadaptive offset filter and an offset value.

The image processing device can further include a deblocking filter unitthat performs deblocking filtering on the image generated by thedecoding unit, and the adaptive offset filter unit can perform adaptiveoffset filtering on an image on which the deblocking filter unit hasperformed deblocking filtering.

The acquisition unit can acquire, from the encoded stream,identification data identifying a parameter of a previous largest codingunit on which adaptive offset filtering has been performed before acurrent largest coding unit as being the same as a parameter of thecurrent largest coding unit, and the adaptive offset filter unit canperform adaptive offset filtering on the image generated by the decodingunit, in units of a largest coding unit using the identification dataacquired by the acquisition unit.

The acquisition unit can acquire, from the encoded stream,identification data identifying whether or not to use a parameter of aprevious largest coding unit on which adaptive offset filtering has beenperformed before a current largest coding unit, and the adaptive offsetfilter unit can perform adaptive offset filtering on the image generatedby the decoding unit, in units of a largest coding unit using theidentification data acquired by the acquisition unit.

The acquisition unit can acquire, from the encoded stream,identification data identifying whether or not to use a copy of aparameter of a previous largest coding unit on which adaptive offsetfiltering has been performed before a current largest coding unit, andthe adaptive offset filter unit can perform adaptive offset filtering onthe image generated by the decoding unit, in units of a largest codingunit using the identification data acquired by the acquisition unit.

The acquisition unit can acquire, from the encoded stream,identification data specifying a largest coding unit that is identicalto a current largest coding unit in terms of a parameter, from withinprevious largest coding units on which adaptive offset filtering hasbeen performed before the current largest coding unit, and the adaptiveoffset filter unit can perform adaptive offset filtering on the imagegenerated by the decoding unit, in units of a largest coding unit usingthe identification data acquired by the acquisition unit.

The parameters of the adaptive offset filter are transmitted at timingof the beginning of a largest coding unit.

The decoding unit can perform a decoding process in units each having ahierarchical structure.

An image processing method of the first aspect of the present disclosureis performed by an image processing device, including acquiringparameters of an adaptive offset filter in units of a largest codingunit from an encoded stream in which the parameters of the adaptiveoffset filter are set using the largest coding unit as a unit oftransmission; performing a decoding process on the encoded stream togenerate an image; and performing adaptive offset filtering on thegenerated image in units of a largest coding unit using the acquiredparameters.

An image processing device of a second aspect of the present disclosureincludes a setting unit that sets parameters of an adaptive offsetfilter using a largest coding unit as a unit of transmission; anadaptive offset filter unit that performs adaptive offset filtering onan image which has been subjected to a local decoding process in a casewhere an image is encoded, in units of a largest coding unit using theparameters set by the setting unit; an encoding unit that performs anencoding process on an image on which the adaptive offset filter unithas performed adaptive offset filtering, and that generates an encodedstream using the image; and a transmission unit that transmits theparameters set by the setting unit and the encoded stream generated bythe encoding unit.

The parameters of the adaptive offset filter can include a type of theadaptive offset filter and an offset value.

The image processing device can further include a deblocking filter unitthat performs deblocking filtering on a locally decoded image, and theadaptive offset filter unit can perform adaptive offset filtering on animage on which the deblocking filter unit has performed deblockingfiltering.

The setting unit can set identification data identifying a parameter ofa previous largest coding unit on which adaptive offset filtering hasbeen performed before a current largest coding unit as being the same asa parameter of the current largest coding unit, and the transmissionunit can transmit the identification data set by the setting unit andthe encoded stream generated by the encoding unit.

The setting unit can set identification data identifying whether or notto use a parameter of a previous largest coding unit on which adaptiveoffset filtering has been performed before a current largest codingunit, and the transmission unit can transmit the identification data setby the setting unit and the encoded stream generated by the encodingunit.

The setting unit can set identification data identifying whether or notto use a copy of a parameter of a previous largest coding unit on whichadaptive offset filtering has been performed before a current largestcoding unit, and the transmission unit can transmit the identificationdata set by the setting unit and the encoded stream generated by theencoding unit.

The setting unit can set identification data specifying a largest codingunit that is identical to a current largest coding unit in terms of aparameter, from within previous largest coding units on which adaptiveoffset filtering has been performed before the current largest codingunit, and the transmission unit can transmit the identification data setby the setting unit and the encoded stream generated by the encodingunit.

The transmission unit can transmit the parameters of the adaptive offsetfilter set by the setting unit at timing of the beginning of a largestcoding unit.

The encoding unit can perform an encoding process in units each having ahierarchical structure.

An image processing method of the second aspect of the presentdisclosure is performed by an image processing device, including settingparameters of an adaptive offset filter using a largest coding unit as aunit of transmission; performing adaptive offset filtering on an imagewhich has been subjected to a local decoding process in a case where animage is encoded, in units of a largest coding unit using the setparameters; performing an encoding process on an image on which adaptiveoffset filtering has been performed, to generate an encoded stream usingthe image; and transmitting the set parameters and the generated encodedstream.

In the first aspect of the present disclosure, parameters of an adaptiveoffset filter are acquired in units of a largest coding unit from anencoded stream in which the parameters of the adaptive offset filter areset using the largest coding unit as a unit of transmission; and adecoding process is performed on the encoded stream to generate animage. Then, adaptive offset filtering is performed on the generatedimage in units of a largest coding unit using the acquired parameters.

In the second aspect of the present disclosure, parameters of anadaptive offset filter are set using a largest coding unit as a unit oftransmission; and adaptive offset filtering is performed on an imagewhich has been subjected to a local decoding process in a case where animage is encoded, in units of a largest coding unit using the setparameters. Then, an encoding process is performed on an image on whichadaptive offset filtering has been performed, to generate an encodedstream using the image; and the set parameters and the generated encodedstream are transmitted.

Note that each of the image processing devices described above may be anindependent device or an internal block included in a single imageencoding device or image decoding device.

According to the first aspect of the present disclosure, it is possibleto decode an image. In particular, it is possible to improve processingefficiency.

According to the second aspect of the present disclosure, it is possibleto encode an image. In particular, it is possible to improve processingefficiency.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a main example configuration ofan image encoding device.

FIG. 2 is a flowchart depicting an example of the flow of an encodingprocess.

FIG. 3 is a block diagram illustrating a main example configuration ofan image decoding device.

FIG. 4 is a flowchart depicting an example of the flow of a decodingprocess.

FIG. 5 is a diagram depicting a quad-tree structure in the HEVC scheme.

FIG. 6 is a diagram depicting band offset.

FIG. 7 is a diagram depicting edge offset.

FIG. 8 is a diagram illustrating classification rules for edge offset.

FIG. 9 is a diagram depicting an overview of the present technology.

FIG. 10 is a diagram depicting advantages of the present technology.

FIG. 11 is a diagram illustrating an example of the syntax of sao_param()

FIG. 12 is a diagram depicting sao_type_idx.

FIG. 13 is a block diagram illustrating an example configuration of anadaptive offset filter to which the present disclosure is applied.

FIG. 14 is a flowchart depicting an adaptive offset filter process.

FIG. 15 is a flowchart depicting a coefficient writing process.

FIG. 16 is a diagram depicting the coefficient writing process.

FIG. 17 is a block diagram illustrating an example configuration of anadaptive offset filter to which the present disclosure is applied.

FIG. 18 is a flowchart depicting an adaptive offset filter process.

FIG. 19 is a flowchart depicting a coefficient reading process.

FIG. 20 is a diagram depicting the coefficient reading process.

FIG. 21 is a diagram illustrating an example of a multi-view imageencoding scheme.

FIG. 22 is a diagram illustrating a main example configuration of amulti-view image encoding device to which the present technology isapplied.

FIG. 23 is a diagram illustrating a main example configuration of amulti-view image decoding device to which the present technology isapplied.

FIG. 24 is a diagram illustrating an example of a layered image encodingscheme.

FIG. 25 is a diagram illustrating a main example configuration of alayered image encoding device to which the present technology isapplied.

FIG. 26 is a diagram illustrating a main example configuration of alayered image decoding device to which the present technology isapplied.

FIG. 27 is a block diagram illustrating a main example configuration ofa computer.

FIG. 28 is a block diagram illustrating an example of a schematicconfiguration of a television apparatus.

FIG. 29 is a block diagram illustrating an example of a schematicconfiguration of a mobile phone.

FIG. 30 is a block diagram illustrating an example of a schematicconfiguration of a recording/reproducing apparatus.

FIG. 31 is a block diagram illustrating an example of a schematicconfiguration of an imaging apparatus.

FIG. 32 is a block diagram illustrating an example of the use ofscalable coding.

FIG. 33 is a block diagram illustrating another example of the use ofscalable coding.

FIG. 34 is a block diagram illustrating still another example of the useof scalable coding.

DESCRIPTION OF EMBODIMENTS

Modes for carrying out the present disclosure (hereinafter referred toas embodiments) will be described hereinafter. Note that the descriptionwill be given in the following order.

1. Overview of devices and operations2. Explanation of related art techniques3. First embodiment (image processing device)4. Second embodiment (multi-view image encoding/multi-view imagedecoding device)5. Third embodiment (layered image encoding/layered image decodingdevice)6. Fourth embodiment (computer)7. Exemplary applications8. Exemplary applications of scalable coding

1. Overview of Devices and Operations [Example Configuration of ImageEncoding Device]

FIG. 1 illustrates a configuration of an embodiment of an image encodingdevice serving as an image processing device to which the presentdisclosure is applied.

An image encoding device 11 illustrated in FIG. 1 encodes image datausing a prediction process. Examples of a coding scheme, as used here,include the HEVC (High Efficiency Video Coding) scheme. In the HEVCscheme, a coding unit CU, a largest coding unit LCU, a smallest codingunit SCU, a prediction unit PU, and a transform unit TU are specified,and encoding/decoding is performed in units each having a hierarchicalstructure.

In the example of FIG. 1, the image encoding device 11 includes an A/D(Analog/Digital) conversion unit 21, a screen rearrangement buffer 22, acomputation unit 23, an orthogonal transform unit 24, a quantizationunit 25, a lossless encoding unit 26, and an accumulation buffer 27. Theimage encoding device 11 further includes a dequantization unit 28, aninverse orthogonal transform unit 29, a computation unit 30, adeblocking filter 31, a frame memory 32, a selection unit 33, an intraprediction unit 34, a motion prediction and compensation unit 35, aprediction image selection unit 36, and a rate control unit 37.

The image encoding device 11 further includes an adaptive offset filter41 and an adaptive loop filter 42 between the deblocking filter 31 andthe frame memory 32.

The A/D conversion unit 21 performs A/D conversion on input image data,and outputs the resulting image data to the screen rearrangement buffer22 for storage.

The screen rearrangement buffer 22 rearranges a stored image havingframes arranged in display order into an image having frames arranged inorder for coding in accordance with a GOP (Group Of Picture) structure.The screen rearrangement buffer 22 supplies an image in which the frameshave been reordered to the computation unit 23. The screen rearrangementbuffer 22 also supplies the image in which the frames have beenreordered to the intra prediction unit 34 and the motion prediction andcompensation unit 35.

The computation unit 23 subtracts a prediction image to be supplied fromthe intra prediction unit 34 or the motion prediction and compensationunit 35 through the prediction image selection unit 36 from the imageread from the screen rearrangement buffer 22, and outputs differenceinformation on the difference therebetween to the orthogonal transformunit 24.

For example, in the case of an image to be intra-coded, the computationunit 23 subtracts a prediction image supplied from the intra predictionunit 34 from the image read from the screen rearrangement buffer 22.Further, for example, in the case of an image to be inter-coded, thecomputation unit 23 subtracts a prediction image supplied from themotion prediction and compensation unit 35 from the image read from thescreen rearrangement buffer 22.

The orthogonal transform unit 24 performs an orthogonal transform, suchas a discrete cosine transform or a Karhunen-Loève transform, on thedifference information supplied from the computation unit 23, andsupplies obtained transform coefficients to the quantization unit 25.

The quantization unit 25 quantizes the transform coefficients outputfrom the orthogonal transform unit 24. The quantization unit 25 suppliesthe quantized transform coefficients to the lossless encoding unit 26.

The lossless encoding unit 26 performs lossless encoding, such asvariable-length coding or arithmetic coding, on the quantized transformcoefficients.

The lossless encoding unit 26 acquires parameters such as informationindicating an intra prediction mode from the intra prediction unit 34,and acquires parameters such as information indicating an interprediction mode and motion vector information from the motion predictionand compensation unit 35.

The lossless encoding unit 26 encodes the quantized transformcoefficients and the acquired respective parameters (syntax elements),and organizes (multiplexes) the encoded quantized transform coefficientsand parameters into part of header information of the encoded data. Thelossless encoding unit 26 supplies the encoded data obtained by encodingto the accumulation buffer 27 for accumulation.

In the lossless encoding unit 26, for example, a lossless encodingprocess such as variable-length coding or arithmetic coding isperformed. Examples of the variable-length encoding include CAVLC(Context-Adaptive Variable Length Coding). Examples of the arithmeticcoding include CABAC (Context-Adaptive Binary Arithmetic Coding).

The accumulation buffer 27 temporarily holds the encoded stream (data)supplied from the lossless encoding unit 26, and outputs the encodedstream (data) to an unillustrated downstream device such as a recordingdevice and a transmission path at certain timing as encoded image whichhas been subjected to encoding. That is, the accumulation buffer 27 alsoserves as a transmission unit that transmits encoded streams.

Further, the transform coefficients quantized by the quantization unit25 are also supplied to the dequantization unit 28. The dequantizationunit 28 dequantizes the quantized transform coefficients using a methodcorresponding to the method of quantization performed by thequantization unit 25. The dequantization unit 28 supplies the obtainedtransform coefficients to the inverse orthogonal transform unit 29.

The inverse orthogonal transform unit 29 performs an inverse orthogonaltransform on the supplied transform coefficients using a methodcorresponding to the orthogonal transform process performed by theorthogonal transform unit 24. The output subjected to the inverseorthogonal transform (restored difference information) is supplied tothe computation unit 30.

The computation unit 30 adds the prediction image supplied from theintra prediction unit 34 or the motion prediction and compensation unit35 through the prediction image selection unit 36 to the result of theinverse orthogonal transform supplied from the inverse orthogonaltransform unit 29, that is, to the restored difference information, toobtain a locally decoded image (decoded image).

For example, if the difference information corresponds to an image to beintra-coded, the computation unit 30 adds the prediction image suppliedfrom the intra prediction unit 34 to the difference information.Further, for example, if the difference information corresponds to animage to be inter-coded, the computation unit 30 adds the predictionimage supplied from the motion prediction and compensation unit 35 tothe difference information.

The decoded image, which is a result of the addition, is supplied to thedeblocking filter 31 and the frame memory 32.

The deblocking filter 31 appropriately performs a deblocking filterprocess to remove block distortion from the decoded image. Thedeblocking filter 31 supplies a result of the filter process to theadaptive offset filter 41.

The adaptive offset filter 41 performs an offset filter (SAO: Sampleadaptive offset) process on the image filtered by the deblocking filter31 to mainly remove ringing.

There are nine offset filter types in total: two types of band offset,six types of edge offset, and no offset. The adaptive offset filter 41determines a kind (type) of offset filter and an offset (value) for eachLCU, which is the largest coding unit, and performs a filter process onthe image filtered by the deblocking filter 31 using the determined typeand offset. In the offset filter 41, the offset described above is thecoefficient of the filter. The offset is hereinafter also referred to asthe coefficient, as necessary.

Note that, as the details of the adaptive offset filter 41 will bedescribed below with reference to FIG. 13, the adaptive offset filter 41has a buffer for storing coefficients. If the buffer has stored thereinthe same coefficient as a coefficient determined for each LCU, theadaptive offset filter 41 supplies a flag indicating the storage of thecoefficient, an index indicating the storage position of the coefficientin the buffer, and information indicating the type to the losslessencoding unit 26 to encode them.

On the other hand, if the buffer does not have stored therein the samecoefficient as a coefficient determined for each LCU, the adaptiveoffset filter 41 supplies a flag indicating no storage of thecoefficient, the coefficient, and information indicating the type to thelossless encoding unit 26 to encode them.

The adaptive offset filter 41 supplies the image that has been subjectedto the filter process to the adaptive loop filter 42.

The adaptive loop filter 42 performs an adaptive loop filter (ALF)process, for example, for each LCU, which is the largest coding unit. Inthe adaptive loop filter 42, for example, a two-dimensional Wienerfilter is used as a filter. As a matter of course, any filter other thana Wiener filter may be used.

The adaptive loop filter 42 performs a filter process on the imagefiltered by the adaptive offset filter 41 for each LCU using a filtercoefficient, and supplies a result of the filter process to the framememory 32.

Note that, in the image encoding device 11, a filter coefficient iscalculated by the adaptive loop filter 42 for each LCU so as to minimizethe residue from the original image from the screen rearrangement buffer12, and is used, which will not be described in detail herein. Thecalculated filter coefficient is encoded by the lossless encoding unit26, and is transmitted to an image decoding device 51 of FIG. 3described below. Furthermore, while an example in which processing isperformed for each LUC is described herein, the processing unit of theadaptive loop filter 42 is not limited thereto.

The frame memory 32 outputs a reference image accumulated therein to theintra prediction unit 34 or the motion prediction and compensation unit35 through the selection unit 33 at certain timing.

For example, in the case of an image to be intra-coded, the frame memory32 supplies a reference image to the intra prediction unit 34 throughthe selection unit 33. Further, for example, in a case where intercoding is to be performed, the frame memory 32 supplies a referenceimage to the motion prediction and compensation unit 35 through theselection unit 33.

If the reference image supplied from the frame memory 32 is an image forintra coding, the selection unit 33 supplies the reference image to theintra prediction unit 34. Further, if a reference image supplied fromthe frame memory 32 is an image for inter coding, the selection unit 33supplies the reference image to the motion prediction and compensationunit 35.

The intra prediction unit 34 performs intra prediction (intra-screenprediction) to generate a prediction image using pixel values in ascreen. The intra prediction unit 34 performs intra prediction using aplurality of modes (intra prediction modes).

The intra prediction unit 34 generates a prediction image in all theintra prediction modes, evaluates each prediction image, and selects anoptimum mode. Upon selection of an optimum intra prediction mode, theintra prediction unit 34 supplies the prediction image generated in theoptimum mode to the computation unit 23 and the computation unit 30through the prediction image selection unit 36.

In addition, as described above, the intra prediction unit 34 suppliesparameters, such as intra-prediction mode information indicating theintra prediction mode that has been adopted, to the lossless encodingunit 26, as necessary.

The motion prediction and compensation unit 35 performs motionprediction on an image to be inter-coded, using the input image suppliedfrom the screen rearrangement buffer 22 and the reference image suppliedfrom the frame memory 32 through the selection unit 33. The motionprediction and compensation unit 35 further performs a motioncompensation process in accordance with motion vectors detected throughmotion prediction, and generates a prediction image (inter predictionimage information).

The motion prediction and compensation unit 35 performs an interprediction process for all the candidate inter prediction modes, andgenerates a prediction image. The motion prediction and compensationunit 35 supplies the generated prediction image to the computation unit23 and the computation unit 30 through the prediction image selectionunit 36.

The motion prediction and compensation unit 35 further suppliesparameters, such as the inter-prediction mode information indicating theinter prediction mode that has been adopted, and motion vectorinformation indicating the calculated motion vectors, to the losslessencoding unit 26.

In the case of an image to be intra-coded, the prediction imageselection unit 36 supplies the output of the intra prediction unit 34 tothe computation unit 23 and the computation unit 30. In the case of animage to be inter-coded, the prediction image selection unit 36 suppliesthe output of the motion prediction and compensation unit 35 to thecomputation unit 23 and the computation unit 30.

The rate control unit 37 controls the rate of the quantization operationof the quantization unit 25 on the basis of the compressed imageaccumulated in the accumulation buffer 27 so that overflow or underflowwill not occur.

[Operation of Image Encoding Device]

The flow of the encoding process executed by the image encoding device11 as described above will be described with reference to FIG. 2.

In step S11, the A/D conversion unit 21 performs A/D conversion on aninput image. In step S12, the screen rearrangement buffer 22 stores theimage subjected to A/D conversion, and rearranges pictures in displayorder into coding order.

If the image to be processed supplied from the screen rearrangementbuffer 22 is an image of a block to be subjected to intra-processing,the decoded image to be referenced is read from the frame memory 32, andis supplied to the intra prediction unit 34 through the selection unit33.

In step S13, based on these images, the intra prediction unit 34performs intra prediction on pixels in the processing target block inall the candidate intra prediction modes. Note that the decoded pixel tobe referenced may be a pixel that is not subjected to filtering by thedeblocking filter 31.

Through the process described above, intra prediction is performed inall the candidate intra prediction modes, and cost functions arecalculated for all the candidate intra prediction modes. Then, anoptimum intra-prediction mode is selected on the basis of the calculatedcost functions, and a prediction image generated through intraprediction in the optimum intra-prediction mode and its cost functionare supplied to the prediction image selection unit 36.

If the image to be processed supplied from the screen rearrangementbuffer 22 is an image to be subjected to inter-processing, an image tobe referenced is read from the frame memory 32, and is supplied to themotion prediction and compensation unit 35 through the selection unit33. In step S14, based on these images, the motion prediction andcompensation unit 35 performs a motion prediction and compensationprocess.

Through the process described above, a motion prediction process isperformed in all the candidate inter prediction modes, and costfunctions are calculated for all the candidate inter prediction modes.An optimum inter-prediction mode is determined on the basis of thecalculated cost functions. Then, a prediction image generated in theoptimum inter-prediction mode and its cost function are supplied to theprediction image selection unit 36.

In step S15, the prediction image selection unit 36 determines one ofthe optimum intra-prediction mode and the optimum inter-prediction modeas an optimum prediction mode on the basis of the respective costfunctions output from the intra prediction unit 34 and the motionprediction and compensation unit 35. Then, the prediction imageselection unit 36 selects a prediction image of the determined optimumprediction mode, and supplies the selected prediction image to thecomputation units 23 and 30. This prediction image is used forcomputation in steps S16 and S21 described below.

Note that selection information on this prediction image is supplied tothe intra prediction unit 34 or the motion prediction and compensationunit 35. If a prediction image of the optimum intra-prediction mode isselected, the intra prediction unit 34 supplies information indicatingthe optimum intra-prediction mode (i.e., parameters related to intraprediction) to the lossless encoding unit 26.

If a prediction image of the optimum inter-prediction mode is selected,the motion prediction and compensation unit 35 outputs informationindicating the optimum inter-prediction mode and informationcorresponding to the optimum inter-prediction mode (i.e., parametersrelated to motion prediction) to the lossless encoding unit 26. Examplesof the information corresponding to the optimum inter-prediction modeinclude motion vector information and reference frame information.

In step S16, the computation unit 23 computes a difference between theimage subjected to rearrangement in step S12 and the prediction imageselected in step S15. The prediction image is supplied to thecomputation unit 23 through the prediction image selection unit 36 fromthe motion prediction and compensation unit 35 for inter prediction orfrom the intra prediction unit 34 for intra prediction.

The difference data has a smaller amount of data than the original imagedata. Accordingly, the amount of data can be reduced compared to thatfor an image that is encoded as it is.

In step S17, the orthogonal transform unit 24 performs an orthogonaltransform on the difference information supplied from the computationunit 23. Specifically, an orthogonal transform such as a discrete cosinetransform or a Karhunen-Loève transform is performed, and transformcoefficients are output.

In step S18, the quantization unit 25 quantizes the transformcoefficients. In this quantization, as described with reference to theprocessing of step S28 described below, the rate is controlled.

The difference information quantized in the way described above islocally decoded in the following way: In step S19, the dequantizationunit 28 dequantizes the transform coefficients quantized by thequantization unit 25, using the characteristics corresponding to thecharacteristics of the quantization unit 25. In step S20, the inverseorthogonal transform unit 29 performs an inverse orthogonal transform onthe transform coefficients dequantized by the dequantization unit 28using the characteristics corresponding to the characteristics of theorthogonal transform unit 24.

In step S21, the computation unit 30 adds a prediction image inputthrough the prediction image selection unit 36 to the locally decodeddifference information, and generates a locally decoded image (an imagecorresponding to the input to the computation unit 23).

In step S22, the deblocking filter 31 performs a deblocking filterprocess on the image output from the computation unit 30. Accordingly,block distortion is removed. The filtered image obtained from thedeblocking filter 31 is output to the adaptive offset filter 41.

In step S23, the adaptive offset filter 41 performs an adaptive offsetfilter process. Through this process, the type and coefficient of theoffset filter are determined for each LCU, which is the largest codingunit, and a filter process is performed on the image filtered by thedeblocking filter 31, using the type and coefficient of the offsetfilter. Note that the details of this adaptive offset filter processwill be described below with reference to FIG. 14.

Then, if the buffer has stored therein the same coefficient as acoefficient determined for each LCU, a flag indicating the storage ofthe coefficient, an index indicating the storage position in the buffer,and information indicating the type are supplied to the losslessencoding unit 26. On the other hand, if the buffer does not have storedtherein the same coefficient as a coefficient determined for each LCU, aflag indicating no storage of the coefficient, the coefficient, andinformation indicating the type are supplied to the lossless encodingunit 26.

These pieces of information supplied to the lossless encoding unit 26(hereinafter collectively referred to as adaptive offset parameters) areencoded in step S26 described below.

In step S24, the adaptive loop filter 42 performs an adaptive loopfilter process on the image filtered by the adaptive offset filter 41.For example, the image filtered by the adaptive offset filter 41undergoes a filter process for each LCU using a filter coefficient, anda result of the filter process on the image is supplied to the framememory 32.

As described above, making the processing unit of the adaptive offsetfilter 41 match the processing unit of the adaptive loop filter 42 canprovide efficient processing.

In step S25, the frame memory 32 stores the filtered image. Note that animage that is not filtered by the deblocking filter 31, the adaptiveoffset filter 41, or the adaptive loop filter 42 is also supplied to theframe memory 32 from the computation unit 30, and is stored in the framememory 32.

On the other hand, the transform coefficients quantized in step S18,described above, are also supplied to the lossless encoding unit 26. Instep S26, the lossless encoding unit 26 encodes the quantized transformcoefficients output from the quantization unit 25 and also encodes thesupplied parameters. That is, a difference image is losslessly encodedusing variable-length coding, arithmetic coding, or the like, and iscompressed.

In step S27, the accumulation buffer 27 accumulates the encodeddifference image (i.e., an encoded stream) as a compressed image. Acompressed image accumulated in the accumulation buffer 27 is read, asnecessary, and is transmitted to the decoder side through a transmissionpath.

In step S28, the rate control unit 37 controls the rate of thequantization operation of the quantization unit 25 on the basis of thecompressed image accumulated in the accumulation buffer 27 so thatoverflow or underflow will not occur.

After the completion of the processing of step S28, the encoding processends.

[Example Configuration of Image Decoding Device]

FIG. 3 illustrates a configuration of an embodiment of an image decodingdevice serving as an image processing device to which the presentdisclosure is applied. The image decoding device 51 illustrated in FIG.3 is a decoding device corresponding to the image encoding device 11 ofFIG. 1.

It is assumed that an encoded stream (data) which has been subjected toencoding by the image encoding device 11 is transmitted to the imagedecoding device 51 corresponding to the image encoding device 11 througha certain transmission path, and is decoded.

As illustrated in FIG. 3, the image decoding device 51 includes anaccumulation buffer 61, a lossless decoding unit 62, a dequantizationunit 63, an inverse orthogonal transform unit 64, a computation unit 65,a deblocking filter 66, a screen rearrangement buffer 67, and a D/Aconversion unit 68. The image decoding device 51 further includes aframe memory 69, a selection unit 70, an intra prediction unit 71, amotion prediction and compensation unit 72, and a selection unit 73.

The image decoding device 51 further includes an adaptive offset filter81 and an adaptive loop filter 82 between the deblocking filter 66 andthe screen rearrangement buffer 67 and between the deblocking filter 66and the frame memory 69.

The accumulation buffer 61 is also a receiving unit for receivingtransmitted encoded data. The accumulation buffer 61 receives andaccumulates transmitted encoded data. The encoded data has beensubjected to encoding by the image encoding device 11. The losslessdecoding unit 62 decodes encoded data read from the accumulation buffer61 at certain timing, using a scheme corresponding to the coding schemeof the lossless encoding unit 26 of FIG. 1.

The lossless decoding unit 62 supplies the decoded parameters such asinformation indicating the intra prediction mode to the intra predictionunit 71, and supplies the parameters such as information indicating theinter prediction mode and motion vector information to the motionprediction and compensation unit 72. The lossless decoding unit 62further supplies the decoded adaptive offset parameters (such as a flagindicating the presence or absence of a coefficient stored in thebuffer, the coefficient, information indicating a type, and an indexindicating the storage position of the coefficient in the buffer) to theadaptive offset filter 81.

The dequantization unit 63 dequantizes the coefficient data(quantization coefficients) obtained by the lossless decoding unit 62through decoding, using a scheme corresponding to the quantizationscheme of the quantization unit 25 of FIG. 1. That is, thedequantization unit 63 dequantizes the quantization coefficients using amethod similar to that of the dequantization unit 28 of FIG. 1 by usingthe quantization parameters supplied from the image encoding device 11.

The dequantization unit 63 supplies the dequantized coefficient data,that is, the orthogonal transform coefficients, to the inverseorthogonal transform unit 64. The inverse orthogonal transform unit 64performs an inverse orthogonal transform on the orthogonal transformcoefficients using the scheme corresponding to the orthogonal transformscheme of the orthogonal transform unit 24 of FIG. 1, and obtainsdecoded residual data corresponding to the residual data that has notbeen orthogonally transformed in the image encoding device 11.

The decoded residual data obtained by the inverse orthogonal transformis supplied to the computation unit 65. The computation unit 65 is alsosupplied with a prediction image from the intra prediction unit 71 orthe motion prediction and compensation unit 72 through the selectionunit 73.

The computation unit 65 adds together the decoded residual data and theprediction image, and obtains decoded image data corresponding to theimage data from which the prediction image has not been subtracted bythe computation unit 23 of the image encoding device 11. The computationunit 65 supplies the decoded image data to the deblocking filter 66.

The deblocking filter 66 appropriately performs a deblocking filterprocess to remove block distortion from the decoded image. Thedeblocking filter 66 supplies a result of the filter process to theadaptive offset filter 81.

The adaptive offset filter 81 performs an offset filter (SAO) process onthe image filtered by the deblocking filter 66 to mainly remove ringing.

The adaptive offset filter 81 performs a filter process on the imagefiltered by the deblocking filter 66 for each LCU, which is the largestcoding unit, using the adaptive offset parameters supplied from thelossless decoding unit 62. The adaptive offset filter 81 supplies theimage that has been subjected to the filter process to the adaptive loopfilter 82.

Note that, as the details of the adaptive offset filter 81 will bedescribed below with reference to FIG. 12 and the following drawings,the adaptive offset filter 81 has a buffer for storing coefficients. Ifthe flag sent from the lossless decoding unit 62 indicates the presenceof a coefficient stored in the buffer, the adaptive offset filter 81reads the coefficient from the buffer by referring to the informationindicating the type and the index indicating the storage position of thecoefficient in the buffer, and performs a filter process using the readcoefficient.

On the other hand, if the flag sent from the lossless decoding unit 62indicates the absence of a coefficient stored in the buffer, theadaptive offset filter 81 performs a filter process using thecoefficient acquired from the lossless decoding unit 62. After that, theadaptive offset filter 81 writes the acquired coefficient to the buffer.

The adaptive loop filter 82 has a basically similar configuration tothat of the adaptive loop filter 42 of the image encoding device 11 ofFIG. 1, and performs an adaptive loop filter process for each LCU, whichis the largest coding unit. The adaptive loop filter 82 performs afilter process on the image filtered by the adaptive offset filter 81for each LCU using a filter coefficient, and supplies a result of thefilter process to the frame memory 69 and the screen rearrangementbuffer 67.

Note that, in the image decoding device 51, a filter coefficient, whichhas been calculated for each LUC and has been encoded by and sent fromthe adaptive loop filter 42 of the image encoding device 11, is decodedby the lossless decoding unit 62 and is used, which will not bedescribed in detail herein.

The screen rearrangement buffer 67 performs rearrangement on the image.That is, the order of the frames rearranged in coding order by thescreen rearrangement buffer 22 of FIG. 1 is changed to the originaldisplay order. The D/A conversion unit 68 performs D/A conversion on theimage supplied from the screen rearrangement buffer 67, and outputs theresulting image to a display (not illustrated) for display.

The output of the adaptive loop filter 82 is further supplied to theframe memory 69.

The frame memory 69, the selection unit 70, the intra prediction unit71, the motion prediction and compensation unit 72, and the selectionunit 73 correspond to the frame memory 32, the selection unit 33, theintra prediction unit 34, the motion prediction and compensation unit35, and the prediction image selection unit 36 of the image encodingdevice 11, respectively.

The selection unit 70 reads an image to be subjected to inter-processingand an image to be referenced from the frame memory 69, and supplies theread images to the motion prediction and compensation unit 72. Further,the selection unit 70 reads an image to be used for intra predictionfrom the frame memory 69, and supplies the read image to the intraprediction unit 71.

The intra prediction unit 71 is supplied with information indicating anintra prediction mode and the like, which are obtained by decoding theheader information, from the lossless decoding unit 62, as appropriate.The intra prediction unit 71 generates a prediction image, based on thisinformation, from the reference image acquired from the frame memory 69,and supplies the generated prediction image to the selection unit 73.

The motion prediction and compensation unit 72 is supplied withinformation obtained by decoding the header information (prediction modeinformation, motion vector information, reference frame information,flag, various parameters, etc.) from the lossless decoding unit 62.

The motion prediction and compensation unit 72 generates a predictionimage, based on these pieces of information supplied from the losslessdecoding unit 62, from the reference image acquired from the framememory 69, and supplies the generated prediction image to the selectionunit 73.

The selection unit 73 selects the prediction image generated by themotion prediction and compensation unit 72 or the intra prediction unit71, and supplies the selected prediction image to the computation unit65.

[Operation of Image Decoding Device]

An example of the flow of the decoding process executed by the imagedecoding device 51 as described above will be described with referenceto FIG. 4.

When a decoding process starts, in step S51, the accumulation buffer 61receives and accumulates a transmitted encoded stream (data). In stepS52, the lossless decoding unit 62 decodes the encoded data suppliedfrom the accumulation buffer 61. I-pictures, P-pictures, and B-pictures,which have been encoded by the lossless encoding unit 26 of FIG. 1, aredecoded.

Before the decoding of the pictures, information on parameters such asmotion vector information, reference frame information, and predictionmode information (the intra prediction mode or the inter predictionmode) is also decoded.

If the prediction mode information is intra-prediction mode information,the prediction mode information is supplied to the intra prediction unit71. If the prediction mode information is inter-prediction modeinformation, the motion vector information and the like corresponding tothe prediction mode information are supplied to the motion predictionand compensation unit 72. In addition, the adaptive offset parametersare also decoded and supplied to the adaptive offset filter 81.

In step S53, the intra prediction unit 71 or the motion prediction andcompensation unit 72 performs a corresponding prediction imagegeneration process in accordance with the prediction mode informationsupplied from the lossless decoding unit 62.

Specifically, if intra-prediction mode information is supplied from thelossless decoding unit 62, the intra prediction unit 71 generates anintra prediction image for the intra prediction mode. Ifinter-prediction mode information is supplied from the lossless decodingunit 62, the motion prediction and compensation unit 72 performs amotion prediction and compensation process of the inter prediction mode,and generates an inter prediction image.

Through the process described above, the prediction image (intraprediction image) generated by the intra prediction unit 71 or theprediction image (inter prediction image) generated by the motionprediction and compensation unit 72 is supplied to the selection unit73.

In step S54, the selection unit 73 selects a prediction image. That is,the prediction image generated by the intra prediction unit 71 or theprediction image generated by the motion prediction and compensationunit 72 is supplied. Accordingly, the supplied prediction image isselected and is supplied to the computation unit 65, so that theprediction image is added to the output of the inverse orthogonaltransform unit 64 in step S57 described below.

In step S52 described above, the transform coefficients decoded by thelossless decoding unit 62 are also supplied to the dequantization unit63. In step S55, the dequantization unit 63 dequantizes the transformcoefficients decoded by the lossless decoding unit 62, using thecharacteristics corresponding to the characteristics of the quantizationunit 25 of FIG. 1.

In step S56, the inverse orthogonal transform unit 29 performs aninverse orthogonal transform on the transform coefficients dequantizedby the dequantization unit 28, using the characteristics correspondingto the characteristics of the orthogonal transform unit 24 of FIG. 1.Accordingly, difference information corresponding to the input of theorthogonal transform unit 24 of FIG. 1 (the output of the computationunit 23) is decoded.

In step S57, the computation unit 65 adds the prediction image selectedin the processing of step S54 described above and input through theselection unit 73 to the difference information. Accordingly, theoriginal image is decoded.

In step S58, the deblocking filter 66 performs a deblocking filterprocess on the image output from the computation unit 65. Accordingly,block distortion is removed. The decoded image sent from the deblockingfilter 66 is output to the adaptive offset filter 81.

In step S59, the adaptive offset filter 81 performs an adaptive offsetfilter process. The adaptive offset filter 81 performs a filter processon the image filtered by the deblocking filter 66, using the adaptiveoffset parameters sent from the lossless decoding unit 62. The adaptiveoffset filter 81 supplies the image that has been subjected to thefilter process to the adaptive loop filter 82.

Note that, as the details of the adaptive offset filter 81 will bedescribed below with reference to FIG. 12 and the following drawings,the adaptive offset filter 81 has a buffer for storing coefficients. Ifthe flag sent from the lossless decoding unit 62 indicates the presenceof a coefficient stored in the buffer, the coefficient is read from thebuffer with reference to the information indicating the type and theindex indicating the storage position of the coefficient in the buffer,and a filter process is performed using the read coefficient.

On the other hand, if the flag sent from the lossless decoding unit 62indicates the absence of a coefficient stored in the buffer, a filterprocess is performed using the coefficient acquired from the losslessdecoding unit 62. After that, the acquired coefficient is written to thebuffer.

In step S60, the adaptive loop filter 82 performs an adaptive loopfilter process on the image filtered by the adaptive offset filter 81.The adaptive loop filter 82 performs a filter process on an input imagefor each LCU, which is the largest coding unit, using a filtercoefficient computed for each LCU, and supplies a result of the filterprocess to the screen rearrangement buffer 67 and the frame memory 69.

In step S61, the frame memory 69 stores the filtered image.

In step S62, the screen rearrangement buffer 67 performs rearrangementon the image after the application of the adaptive loop filter 82. Thatis, the order of the frames rearranged by the screen rearrangementbuffer 22 of the image encoding device 11 for coding is changed to theoriginal display order.

In step S63, the D/A conversion unit 68 performs D/A conversion on theimage sent from the screen rearrangement buffer 67. This image is outputto a display (not illustrated), and an image is displayed.

After the completion of the processing of step S63, the decoding processends.

<Explanation of Related Art Techniques> [Adaptive Offset Process in HEVCScheme]

Next, an adaptive offset filter in the HEVC scheme will be described. Inthe HEVC scheme, the Sample Adaptive Offset scheme is adopted.

On the encoder side, the adaptive offset filter 41 is disposed betweenthe deblocking filter (DB) 31 and the adaptive loop filter (ALF) 42.Also on the decoder side, the adaptive offset filter 81 is disposedbetween the deblocking filter (DB) 66 and the adaptive loop filter (ALF)82.

The adaptive offset types (kinds) include two types of offset calledband offset, and six types of offset called edge offset, and noapplication of offset is also possible. Furthermore, it is possible topartition an image into quad-tree regions and to select, for eachregion, which of the adaptive offset types described above to use forcoding.

This selection information is coded as PQAO Info. by a coding unit(Entropy Coding), a bit stream is generated, and the generated bitstream is transmitted to the decoder side. Using this method, codingefficiency can be improved.

Here, a quad-tree structure will be described with reference to FIG. 5.

For example, on the encoder side, as indicated by A1 in FIG. 5, a costfunction J0 of Level-0 (a partition depth of 0) indicating that a region0 is not partitioned is computed. Further, cost functions J1, J2, J3,and J4 of Level-1 (a partition depth of 0) indicating that the region 0is partitioned into four regions 1 to 4 are computed.

Then, as indicated by A2, the cost functions are compared, and thepartition regions (Partitions) of Level-1 are selected in accordancewith J0>(J1+J2+J3+J4).

Similarly, as indicated by A3, cost functions J5 to J20 of Level-2 (apartition depth of 2) indicating that the region 0 is partitioned into16 regions 5 to 20 are computed.

Then, as indicated by A4, the cost functions are compared, and thepartition region (Partition) of Level-1 is selected in the region 1 inaccordance with J1<(J5+J6+J9+J10). In the region 2, the partitionregions (Partitions) of Level-2 are selected in accordance withJ2>(J7+J8+J11+J12). In the region 3, the partition regions (Partitions)of Level-2 are selected in accordance with J3>(J13+J14+J17+J18). In theregion 4, the partition region (Partitions) of Level-1 is selected inaccordance with J4>(J15+J16+J19+J20).

As a result, final quad-tree regions (Partitions) indicated by A4 in thequad-tree structure are determined. Then, cost functions for all thetypes, namely, two types of band offset, six types of edge offset, andno offset, are calculated for each of the determined regions in thequad-tree structure, and which offset to use for coding is determined.

For example, in the example of FIG. 5, as indicated by the white arrow,EO(4), that is, the fourth type among the types of edge offset, isdetermined for the region 1. For the region 7, OFF, that is, no offset,is determined. For the region 8, EO(2), that is, the second type amongthe types of edge offset, is determined. For the regions 11 and 12, OFF,that is, no offset, is determined.

Further, for the region 13, BO(1), that is, the first type among thetypes of band offset, is determined. For the region 14, EO(2), that is,the second type among the types of edge offset, is determined. For theregion 17, BO(2), that is, the second type among the types of bandoffset, is determined. For the region 18, BO(1), that is, the first typeamong the types of band offset, is determined. For the region 4, EO(1),that is, the first type among the types of edge offset, is determined.

Next, the details of band offset will be described with reference toFIG. 6.

In band offset, in the example of FIG. 6, each scale represents oneband=8 pixels, and luminance pixel values are separated into 32 bands,each band having an individual offset value.

That is, in the example of FIG. 6, the center 16 bands out of the 0th to255th pixels (32 bands) are grouped in a first group, and the 8 bands ateither side are grouped in a second group.

Then, the offsets in only either the first group or the second group areencoded and are sent to the decoder side. In general, each region isoften a high-contrast white and black region or a low-contrast tintregion, and it is rare that both the first group and the second groupall contain pixels. For this reason, sending the offsets in only onegroup can suppress an increase in the amount of coding which is causedby the transmission of the pixel values of the values that are notincluded in each quad-tree region.

Note that if an input signal is broadcasted, the value of the luminancesignal is limited to the range of 16,235, and the values of thechrominance signals are limited to the range of 16,240. In this case,the broadcast-legal given in the lower part of FIG. 6 is applied, andthe offset values for 2 bands at either side, which are marked by across, are not transmitted.

Next, the details of edge offset will be described with reference toFIG. 7.

In edge offset, a comparison is made between the target pixel value anda neighboring pixel value adjacent to the target pixel value, and anoffset value is transmitted in accordance with the correspondingcategory.

Edge offset has four one-dimensional patterns illustrated in part A ofFIG. 7 to part D of FIG. 7, and two two-dimensional patterns illustratedin part E of FIG. 7 and part F of FIG. 7, and offsets for the categoriesillustrated in FIG. 7 are transmitted.

Part A of FIG. 7 illustrates a 1-D 0-degree pattern in which neighboringpixels are arranged one-dimensionally to the right and left of thetarget pixel C, that is, a 1-D 0-degree pattern which defines an angleof 0 degrees with the pattern in part A of FIG. 7. Part B of FIG. 7illustrates a 1-D 90-degree pattern in which neighboring pixels arearranged one-dimensionally above and below the target pixel C, that is,a 1-D 90-degree pattern which defines an angle of 90 degrees with thepattern in part A of FIG. 7.

Part C of FIG. 7 illustrates a 1-D 135-degree pattern in whichneighboring pixels are arranged one-dimensionally at positions upperleft and lower right from the target pixel C, that is, a 1-D 135-degreepattern which defines an angle of 135 degrees with the pattern in part Aof FIG. 7. Part D of FIG. 7 illustrates a 1-D 135-degree pattern inwhich neighboring pixels are arranged one-dimensionally to the upperright and lower left of the target pixel C, that is, a 1-D 135-degreepattern which defines an angle of 45 degrees with the pattern in part Aof FIG. 7.

Part E of FIG. 7 illustrates a 2-D cross pattern in which neighboringpixels are arranged two-dimensionally above and below and to the rightand left of the target pixel C, that is, a 2-D cross pattern whichcrosses the target pixel C. Part F of FIG. 7 illustrates a 2-D diagonalpattern in which neighboring pixels are arranged two-dimensionally tothe upper right and lower left and to the upper left and lower right ofthe target pixel C, that is, a 2-D diagonal pattern which diagonallycrosses the target pixel C.

Part A of FIG. 8 illustrates classification rules for one-dimensionalpatterns (Classification rule for 1-D patterns). The patterns in part Aof FIG. 7 to part D of FIG. 7 are classified into five categories asillustrated in part A of FIG. 8. An offset is calculated in accordancewith the categories, and is sent to the decoding unit.

If the pixel value of the target pixel C is smaller than the pixelvalues of two neighboring pixels, the pattern is classified incategory 1. If the pixel value of the target pixel C is smaller than thepixel value of one neighboring pixel and is equal to the pixel value ofthe other neighboring pixel, the pattern is classified in category 2. Ifthe pixel value of the target pixel C is larger than the pixel value ofone neighboring pixel and is equal to the pixel value of the otherneighboring pixel, the pattern is classified in category 3. If the pixelvalue of the target pixel C is larger than the pixel values of the twoneighboring pixels, the pattern is classified in category 4. Otherwise,the pattern is classified in category 0.

Part B of FIG. 8 illustrates classification rules for two-dimensionalpatterns (Classification rule for 2-D patterns). The patterns in part Eof FIG. 7 and part F of FIG. 7 are classified into seven categories asillustrated in part B of FIG. 8. An offset is sent to the decoder sidein accordance with the categories.

If the pixel value of the target pixel C is smaller than the pixelvalues of four neighboring pixels, the pattern is classified incategory 1. If the pixel value of the target pixel C is smaller than thepixel values of three neighboring pixel and is equal to the pixel valueof the fourth neighboring pixel, the pattern is classified in category2. If the pixel value of the target pixel C is smaller than the pixelvalues of three neighboring pixel and is larger than the pixel value ofthe fourth neighboring pixel, the pattern is classified in category 3.

If the pixel value of the target pixel C is larger than the pixel valuesof three neighboring pixel and is smaller than the pixel value of thefourth neighboring pixel, the pattern is classified in category 4. Ifthe pixel value of the target pixel C is larger than the pixel values ofthree neighboring pixel and is equal to the pixel value of the fourthneighboring pixel, the pattern is classified in category 5. If the pixelvalue of the target pixel C is larger than the pixel values of the fourneighboring pixels, the pattern is classified in category 6. Otherwise,the pattern is classified in category 0.

As described above, in edge offset, one-dimensional patterns require asmaller amount of computation because a comparison is merely madebetween neighboring two pixels. Note that in a high efficiency codingcondition, a 1-bit offset value is made more accurate than that in a lowdelay coding condition and is sent to the decoder side.

The adaptive offset process described above is a process performed inthe HEVC scheme for each of the determined regions in the quad-treestructure. That is, in the adaptive offset process, it is necessary touniquely define regions for the adaptive offset filter, called aquad-tree structure.

In addition, the parameters of the adaptive offset filter arecollectively defined in sao_param( ) which is placed before the data(video information), in groups of one frame. On the encoder side,therefore, it is necessary to hold data for one frame in a buffer untilthe completion of the adaptive offset filter process, the determinationof the coefficients of the adaptive offset filter, and the creation ofsao_param( ).

In light of the foregoing, in some embodiments, as described in thelater section with reference to FIG. 9, adaptive offset filtering isperformed in units of an LCU, which is the largest coding unit,correspondingly to the LCU-by-LCU processing of adaptive loop filter,which was proposed in NPL 2.

3. First Embodiment Overview of Present Technology

In the example of FIG. 9, an image that is partitioned into LCU 111 toLCU 117 is illustrated. The adaptive offset filter 41 on the encoderside determines the type and coefficient (offset value) of adaptiveoffset filter in units of an LCU, which is the largest coding unit, andsends the type and coefficient to the decoder side at the timing of thebeginning of each LCU. In this case, if the coefficient matches any ofthe coefficients that have already been transmitted and have been storedin the buffer, the adaptive offset filter 81 on the decoder side uses acopy of the coefficient in the buffer.

For example, the type E0 of the LCU 111 (the edge offset of category 0)and its coefficient are determined on the encoder side, and the type andthe coefficient of the LCU 111 are sent to the decoder side at thebeginning of the LCU 111. The type B1 of the LCU 112 (the band offset ofcategory 1) and its coefficient are determined on the encoder side, andthe type and the coefficient of the LCU 111 are sent to the decoder sideat the beginning of the LCU 112. The type E1 of the LCU 113 (the edgeoffset of category 1) and its coefficient are determined on the encoderside, and the type and the coefficient of the LCU 111 are sent to thedecoder side at the beginning of the LCU 113. The type B2 of the LCU 114(the band offset of category 2) and its coefficient are determined onthe encoder side, and the type and the coefficient of the LCU 111 aresent to the decoder side at the beginning of the LCU 114.

On the decoder side, the adaptive offset filter 81 has an EO (edgeoffset) buffer 121 and a BO (band offset) buffer 122, and performs afilter process in units of an LCU. In addition, the transmittedcoefficients are stored.

Here, the type E0 of the LCU 115 (the edge offset of category 0) and itscoefficient are determined on the encoder side. The coefficient of thetype E0 has already been sent to the decoder side at the time of the LCU111, and has been stored in the BO (edge offset) buffer 121 included inthe adaptive offset filter 81 on the decoder side.

For the LCU 115, therefore, a copy of the coefficient of the type E0 inthe EO buffer 121 is used on the decoder side without the sending of thecoefficient of the type E0. Note that, for the LCU 116, no adaptiveoffset (off) is determined on the encoder side, and thus no coefficientis used.

Furthermore, the type E0 of the LOU 117 (the edge offset of category 0)and its coefficient are determined on the encoder side. The coefficientof the type E0 has already been sent to the decoder side at the time ofthe LCU 111, and has been stored in the EO (edge offset) buffer 121included in the adaptive offset filter 81 on the decoder side.

For the LOU 117, therefore, a copy of the coefficient of the type E0 inthe EO buffer 121 is used on the decoder side without the sending of thecoefficient of the type E0.

As described above, the parameters of the adaptive offset filter, whichare transmitted in one batch at the beginning of a frame in the relatedart, are sent sequentially at the beginning of each LCU. Thus, whereas abuffer with a capacity corresponding to one frame is required in therelated art, a buffer with a reduced capacity corresponding to an LCUcan be used.

In addition, a coefficient that has already been used is not sent, thusreducing the increase in the number of sent coefficients, which iscaused by sending a coefficient for each LCU.

FIG. 10 illustrates examples of partitioning of the adaptive offsetfilter. The left part of FIG. 10 illustrates an example ofquad-tree-based partitioning of the related art, and the right part ofFIG. 10 illustrates an example of LCU-based partitioning of the presenttechnology.

In the case of a quad-tree structure, as illustrated in the right partof FIG. 10, an image can only be partitioned into square regions. Incontrast, in the LCU-based filter, as illustrated in the right part ofFIG. 10, an image can be partitioned into convex-shaped regions orconcave-shaped regions.

In actuality, as indicated by a dotted line, a convex-shaped region iscomposed of, for example, an 8×8 LCU and a 16×16 LCU, where the sametype and the use of the same coefficient are depicted. Similarly, asindicated by a dotted line, a concave-shaped region is composed of, forexample, three 8×8 LCUs, where the same type and the use of the samecoefficient are depicted.

As described above, a region is split in units of an LCU, thusincreasing flexibility in partitioning, compared to that based on aquad-tree structure in the related art. Adaptive offset filtering can beperformed in accordance with the characteristics of an input image.

Furthermore, the processing unit of the adaptive offset filter and theprocessing unit of the adaptive loop filter are made to match, allowingboth filter processes to be performed concurrently, in parallel, inpipeline fashion, or the like. Accordingly, processing efficiency can beincreased.

[Example of Syntax]

FIG. 11 is a diagram illustrating an example of the syntax of sao_param() generated by the image encoding device 11. The numerals appearing atthe left side of the respective rows are row numbers given for thepurpose of description.

In the example of FIG. 11, sample_adaptive_offset_flag in the eighth rowis a flag of whether or not to perform adaptive offset filtering.sao_flag_cb in the tenth row indicates the application of the adaptiveoffset filter of cb when it is equal to 1. sao_flag_cr in the twelfthrow indicates the application of the adaptive offset filter of cr whenit is equal to 1. Note that sample_adaptive_offset_flag, sao_flag_cb,and sao_flag_cr are not described herein in particular.

sao_type_idx in the eighteenth row refers to an offset type. copy_flagin the twentieth row specifies whether or not to copy sao_type_idx(type) and sao_offset (coefficient) of the preceding LCU. That is,copy_flag is identification data identifying the parameters (type andoffset) of a preceding LCU on which adaptive offset filtering has beenperformed before the current LCU being processed as being the same asthe parameters of the current LCU. In other words, furthermore,copy_flag is identification data identifying whether to use theparameters of the preceding LCU as the parameters of the current LCU. Inother words, furthermore, copy_flag is identification data identifyingwhether to use a copy of the parameters of the preceding LCU as theparameters of the current LCU.

copy_idx in the twenty-second row refers to previous (processed)sao_type_idx (type) and sao_offset (coefficient) which are to be copied.That is, copy_idx is identification data specifying the same LCU (to becopied) as the current LCU in terms of parameters from within previousLCUs.

sao_offset in the twenty-fifth row refers to the offset (coefficient) ofeach category.

[Example of Type Index]

FIG. 12 is a diagram depicting sao_type_idx. In the example of FIG. 12,sao_type_idx, NumSaoCategory, and Edge or Band type are given insequence from left to right.

sao_type_idx is an index indicating a type of adaptive offset filter.NumSaoCategory is information indicating the number of categories forthe index. Edge or Band type is information indicating what edge (orband) type the index corresponds to. Note that, roughly, sao_type_idx isEdge Offset (EO) when the value of sao_type_idx is in the range of 1 to4, and sao_type_idx is Band Offset (BO) when the value of sao_type_idxis in the range of 5 and 6.

That is, if the value of sao_type_idx is 0, NumSaoCategoriy (the numberof categories) is 0 and an adaptive offset process is not performed.

If the value of sao_type_idx is 1, NumSaoCategoriy (the number ofcategories) is 4 and the type of edge is the 1D 0-degree edgeillustrated in part A of FIG. 7. If the value of sao_type_idx is 2,NumSaoCategoriy (the number of categories) is 4 and the type of edge isthe 1D 90-degree edge illustrated in part B of FIG. 7. If the value ofsao_type_idx is 3, NumSaoCategoriy (the number of categories) is 4 andthe type of edge is the 1D 135-degree edge illustrated in part C of FIG.7. If the value of sao_type_idx is 4, NumSaoCategoriy (the number ofcategories) is 4 and the type of edge is the 1D 45-degree edgeillustrated in part D of FIG. 7.

If the value of sao_type_idx is 5, NumSaoCategoriy (the number ofcategories) is 16 and the type of band is Center Band (the first groupin FIG. 6). If the value of sao_type_idx is 6, NumSaoCategoriy (thenumber of categories) is 16 and the type of band is Side Band (thesecond group in FIG. 6).

Note that while two-dimensional patterns of edge offset are notillustrated in the example of FIG. 7, a two-dimensional pattern of edgeoffset can also be handled by increasing the value of sao_type_idx.

[Example Configuration of Adaptive Offset Filter]

FIG. 13 is a block diagram illustrating an example configuration of theadaptive offset filter and the lossless encoding unit in the imageencoding device of FIG. 1.

In the example of FIG. 13, the adaptive offset filter 41 is constructedto include a type and offset determination unit 211, an offsetprocessing unit 212, an image buffer 213, a coefficient reading unit214, and an offset buffer 215. The adaptive offset filter 41 isconstructed to further include a parameter setting unit 216 and acoefficient writing unit 217.

The lossless encoding unit 26 is constructed to include at least asyntax writing unit 221.

A deblocked pixel value sent from the deblocking filter 31 is input tothe type and offset determination unit 211 and the offset processingunit 212. The type and offset determination unit 211 determines a typeof adaptive offset filter and the offset of the type in units of an LCU.In this case, by way of example, a cost function is computed in units ofan LCU, and the type and offset that are optimum for each LCU, where thecost function is minimum, are determined. The type and offsetdetermination unit 211 supplies a type index (sao_type_idx) indicatingthe determined type and the offset (sao_offset) to the offset processingunit 212 and the coefficient reading unit 214.

Note that the offset (coefficient) is a coefficient of 5 for edgeoffset, and a coefficient of 9 for band offset.

The offset processing unit 212 performs an adaptive offset filterprocess on deblocked pixel values sent from the deblocking filter 31,for each LCU using the type and offset indicated by the type index sentfrom the type and offset determination unit 211. The offset processingunit 212 supplies the pixel values that have been subjected to theoffset process to the image buffer 213.

The image buffer 213 temporarily stores the pixel values that have beensubjected to the offset process by the offset processing unit 212, andsupplies the pixel values to the adaptive loop filter 42 at certaintiming.

The coefficient reading unit 214 searches the offset buffer 215 for anoffset that matches the offset sent from the type and offsetdetermination unit 211. Note that the coefficient reading unit 214checks the buffer of the type (edge offset buffer or band offset buffer)indicated by the type index sent from the offset buffer 215.

If there is any matching coefficient, the coefficient reading unit 214supplies a copy index (copy_idx) indicating the position at which thematching coefficient is stored in the offset buffer 215 to the parametersetting unit 216 together with the type index. If there is no matchingcoefficient, the coefficient reading unit 214 supplies the offset andthe type index to the parameter setting unit 216.

The offset buffer 215 has a buffer for storing offsets for edge offset,and a buffer for storing offsets for band offset. The offset buffer 215is constructed in, for example, a FIFO.

If the copy index is supplied together with the type index, theparameter setting unit 216 sets the copy flag (copy_flag) to ture. Theparameter setting unit 216 supplies the copy flag, the type index, andthe copy index to the syntax writing unit 221 as adaptive offsetparameters.

If the copy index is supplied to the parameter setting unit 216 togetherwith the offset, the parameter setting unit 216 sets the copy flag(copy_flag) to false. The parameter setting unit 216 supplies the copyflag, the type index, and the offset to the syntax writing unit 221 asadaptive offset parameters. In this case, the parameter setting unit 216also supplies the type index and the offset to the coefficient writingunit 217, and causes the offset to be written to an available region inthe offset buffer 215.

The coefficient writing unit 217 writes the offset sent from theparameter setting unit 216 to an available region that is close to thebeginning among available regions in the buffer of the type indicated bythe type index sent from the parameter setting unit 216 within theoffset buffer 215. Note that the available region that is close to thebeginning is an available region having the lowest index.

The syntax writing unit 221 writes the adaptive offset parameters sentfrom the parameter setting unit 216 to the header portion (sao_param) ofthe encoded stream in units of an LCU, as described above with referenceto, for example, FIG. 11.

[Adaptive Offset Filter Process on Encoder Side]

Next, an adaptive offset filter process performed by the adaptive offsetfilter 41 of FIG. 13 will be described with reference to a flowchart ofFIG. 14. Note that this adaptive offset filter process is the process instep S23 of FIG. 2.

A deblocked pixel value sent from the deblocking filter 31 is input tothe type and offset determination unit 211 and the offset processingunit 212. In step S211, the type and offset determination unit 211determines a type of adaptive offset filter and the offset of the typein units of an LCU. The type and offset determination unit 211 suppliesa type index (sao_type_idx) indicating the determined type and theoffset (sao_offset) to the coefficient reading unit 214.

The coefficient reading unit 214 searches the offset buffer 215 for anoffset that matches the offset sent from the type and offsetdetermination unit 211. In step S212, the coefficient reading unit 214determines whether or not there is a matching offset in the offsetbuffer 215. Note that the coefficient reading unit 214 checks the bufferof the type (edge offset buffer or band offset buffer) indicated by thetype index sent from the offset buffer 215.

If it is determined in step S212 that there is no matching offset, thecoefficient reading unit 214 supplies the offset and the type index tothe parameter setting unit 216.

In response to this, in step S213, the parameter setting unit 216 setsthe copy flag (copy_flag) to false.

The parameter setting unit 216 also supplies the type index and theoffset to the coefficient writing unit 217, and, in step S214, causes acoefficient (offset) writing process to be performed. This coefficientwriting process will be described below with reference to FIG. 15.

In step S217, the parameter setting unit 216 supplies the copy flag, thetype index, and the offset to the syntax writing unit 221 as adaptiveoffset parameters, and causes the adaptive offset parameters to beencoded.

In response to this, in step S26 of FIG. 2 described above, the syntaxwriting unit 221 writes the adaptive offset parameters sent from theparameter setting unit 216 to the header portion (sao_param) of theencoded stream in units of an LCU.

On the other hand, if it is determined in step S212 that there is amatching offset, the process proceeds to step S215.

In step S215, the coefficient reading unit 214 sets a copy index(copy_idx) indicating the position at which the matching coefficient isstored in the offset buffer 215, and supplies the copy index (copy_idx)to the parameter setting unit 216 together with the type index.

In response to this, in step S216, the parameter setting unit 216 setsthe copy flag (copy_flag) to ture.

Then, the parameter setting unit 216 supplies the copy flag, the typeindex, and the copy index to the syntax writing unit 221 as adaptiveoffset parameters, and causes the adaptive offset parameters to beencoded.

In response to this, in step S26 of FIG. 2 described above, the syntaxwriting unit 221 writes the adaptive offset parameters sent from theparameter setting unit 216 to the header portion (sao_param) of theencoded stream in units of an LCU.

Meanwhile, the type index (sao_type_idx) and offset (sao_offset)determined in step S211 are also supplied to the offset processing unit212.

In step S218, the offset processing unit 212 performs an offset process.That is, the offset processing unit 212 performs an adaptive offsetfilter process on deblocked pixel values sent from the deblocking filter31, for each LCU using the type and offset indicated by the type indexsent from the type and offset determination unit 211. The pixel valuesthat have been subjected to the adaptive offset filter are stored in theimage buffer 213, and are supplied to the adaptive loop filter 42 atcertain timing.

[Coefficient Writing Process]

Next, the coefficient writing process in step S214 of FIG. 14 will bedescribed with reference to a flowchart of FIG. 15. Note that FIG. 16 isreferred to for the description of the coefficient writing process.

As illustrated in FIG. 16, the offset buffer 215 includes two buffers,that is, an EO buffer 215-1 in which coefficients for edge offset arestored, and a BO buffer 215-2 in which coefficients for band offset arestored.

The EO buffer 215-1 and the BO buffer 215-2 store coefficients, startingfrom the position that is close to the beginning (left side). In the EObuffer 215-1 and the BO buffer 215-2, a position marked with copy index(idx) is a region where a coefficient has already been written, and aposition marked with “empty” is an available region.

That is, in the EO buffer 215-1, coefficients have already been writtento the position marked with copy index (idx)=1, the position marked withcopy index=2, the position marked with copy index=3, and the positionmarked with copy index=4.

For example, in the BO buffer 215-2, coefficients have already beenwritten to the position marked with copy index (idx)=1 and the positionmarked with copy index=2.

In step S231 of FIG. 15, the coefficient writing unit 217 determineswhether or not the type index sent from the parameter setting unit 216is EO (edge offset).

If it is determined in step S231 that the type index is EO (edgeoffset), in step S232, the coefficient writing unit 217 selects the EObuffer 215-1.

If it is determined in step S231 that the type index is not EO (edgeoffset), that is, the type index is band offset, in step S233, thecoefficient writing unit 217 selects the BO buffer 215-2.

In step S234, the coefficient writing unit 217 writes sao_offset(offset) to the available region that is close to the beginning in theselected buffer.

For example, as illustrated in FIG. 16, if the type index (sao_type_idx)is 5, the type index indicates band offset. Thus, in step S233, the BObuffer 215-2 is selected. Then, in step S234, sao_offset (offset) with atype index of 5 is written to the position right adjacent to theposition marked with copy index=2. That is, the position right adjacentto the position marked with copy index=2 is the region that is close tothe beginning among available regions.

[Example Configuration of Adaptive Offset Filter]

FIG. 17 is a block diagram illustrating an example configuration of thelossless decoding unit and the adaptive offset filter in the imagedecoding device of FIG. 3.

In the example of FIG. 17, the lossless decoding unit 62 is constructedto include at least a syntax reading unit 251.

The adaptive offset filter 81 is constructed to include a parameterreceiving unit 261, a coefficient reading unit 262, an offset buffer263, a coefficient writing unit 264, an offset processing unit 265, andan image buffer 266.

The syntax reading unit 251 reads syntax from the header portion of theencoded stream, and supplies adaptive offset parameters in the syntax tothe parameter receiving unit 261.

Note that, as also described above with reference to the encoder side,if the copy flag is ture, the adaptive offset parameters include a copyflag, a type index, and a copy index. If the copy flag is false, theadaptive offset parameters include a copy flag, a type index, and anoffset.

The parameter receiving unit 261 receives adaptive offset parameterssupplied from the syntax reading unit 251, and supplies the adaptiveoffset parameters to the coefficient reading unit 262.

The coefficient reading unit 262 determines whether or not the typeindex is 0. If the type index is 0, the coefficient reading unit 262does not cause the offset processing unit 265 to perform adaptive offsetfiltering.

If the copy flag is true, the coefficient reading unit 262 reads theoffset from the position indicated by the copy index of the buffer ofthe type indicated by the type index. The coefficient reading unit 262supplies the read offset and type index to the offset processing unit265.

If the copy flag is false, the coefficient reading unit 262 supplies theoffset and the type index to the offset processing unit 265. In thiscase, the coefficient reading unit 262 also supplies the offset and thetype index to the coefficient writing unit 264, and causes the offset tobe written to the offset buffer 263.

The offset buffer 263 has a configuration similar to that of the offsetbuffer 215 of FIG. 13. That is, the offset buffer 263 has a buffer forstoring offsets for edge offset, and a buffer for storing offsets forband offset. The offset buffer 263 is constructed in, for example, aFIFO.

The coefficient writing unit 264 has a basically similar configurationto that of the coefficient writing unit 217 of FIG. 13. That is, thecoefficient writing unit 264 writes the offset sent from the coefficientreading unit 262 to an available region that is close to the beginningamong available regions in the buffer of the type indicated by the typeindex sent from the coefficient reading unit 262 within the offsetbuffer 263.

The offset processing unit 265 has a basically similar configuration tothat of the offset processing unit 212 of FIG. 13. That is, the offsetprocessing unit 265 performs an adaptive offset filter process ondeblocked pixel values sent from the deblocking filter 66, for each LCUusing the type and offset indicated by the type index sent from thecoefficient reading unit 262. The offset processing unit 265 suppliesthe pixel values that have been subjected to the offset process to theimage buffer 266.

Note that if the type index is 0, the offset processing unit 265supplies the filtered pixels sent from the deblocking filter 66 to theimage buffer 266 as they are. That is, in this case, an adaptive offsetfilter process is not performed.

The image buffer 266 has a basically similar configuration to that ofthe image buffer 213 of FIG. 13. That is, the image buffer 266temporarily stores the pixel values that have been subjected to theoffset process by the offset processing unit 265, and supplies the pixelvalues to the adaptive loop filter 82 at certain timing.

[Adaptive Offset Filter Process on Decoder Side]

Next, the adaptive offset filter process performed by the adaptiveoffset filter 81 of FIG. 17 will be described with reference to aflowchart of FIG. 18. Note that this adaptive offset filter process isthe process in step S59 of FIG. 4.

In step S52 of FIG. 4, when an encoded stream is to be decoded, thesyntax reading unit 251 reads syntax from the header portion of theencoded stream, and supplies adaptive offset parameters in the syntax tothe parameter receiving unit 261.

In step S251, the parameter receiving unit 261 receives the adaptiveoffset parameters supplied from the syntax reading unit 251, andsupplies the adaptive offset parameters to the coefficient reading unit262.

In step S252, the coefficient reading unit 262 determines whether or notthe type index (sao_type_idx) is 0.

If it is determined in step S252 that the type index is 0, the adaptiveoffset filter process ends. That is, in this case, an adaptive offsetfilter process is not performed, and the offset processing unit 265supplies the filtered pixels sent from the deblocking filter 66 to theimage buffer 266 as they are.

If it is determined in step S252 that the type index is not 0, in stepS253, the coefficient reading unit 262 determines whether or not thecopy flag (copy_flag) is true.

If it is determined in step S253 that the copy flag is ture, in stepS254, the coefficient reading unit 262 performs a coefficient (offset)reading process. This coefficient reading process will be describedbelow with reference to FIG. 19.

Through the process described above, the offset is read from theposition indicated by the copy index of the buffer of the type indicatedby the type index. The coefficient reading unit 262 supplies the readoffset and the type index to the offset processing unit 265.

On the other hand, if it is determined in step S253 that the copy flagis not ture, the coefficient reading unit 262 supplies the offset andtype index sent from the parameter receiving unit 261 to the offsetprocessing unit 265.

Then, in step S255, the coefficient reading unit 262 also supplies theoffset and type index sent from the parameter receiving unit 261 to thecoefficient writing unit 264, and causes a coefficient writing processfor the offset buffer 263 to be performed. This coefficient writingprocess is basically the same process as the coefficient writing processdescribed above with reference to FIG. 15, and a description thereof isthus omitted.

In step S256, the offset processing unit 265 performs an adaptive offsetfilter process on deblocked pixel values sent from the deblocking filter66, for each LCU using the type and offset indicated by the type indexsent from the coefficient reading unit 262. The offset processing unit265 supplies the pixel values that have been subjected to the offsetprocess to the image buffer 266.

[Coefficient Reading Process]

Next, the coefficient reading process in step S254 of FIG. 18 will bedescribed with reference to a flowchart of FIG. 19. Note that FIG. 20 isreferred to for the description of the coefficient reading process.

As illustrated in FIG. 20, the offset buffer 263 includes two buffers,that is, an EO buffer 263-1 in which coefficients for edge offset arestored, and a BO buffer 263-2 in which coefficients for band offset arestored.

The EO buffer 263-1 and the BO buffer 263-2 store coefficients, startingfrom the position that is close to the beginning (left side). In the EObuffer 263-1 and the BO buffer 263-2, a position marked with copy index(idx) is a region where a coefficient has already been written, and theother positions are available regions.

That is, in the EO buffer 263-1, coefficients have already been writtento the position marked with copy index (idx)=1, the position marked withcopy index=2, the position marked with copy index=3, and the positionmarked with copy index=4.

For example, in the BO buffer 263-2, coefficients have already beenwritten to the position marked with copy index (idx)=1, the positionmarked with copy index=2, the position marked with copy index=3, andposition marked with copy index=4.

In step S271 of FIG. 19, the coefficient reading unit 262 determineswhether or not the type index sent from the parameter receiving unit 261is EO (edge offset).

If it is determined in step S271 that the type index is EO (edgeoffset), in step S272, the coefficient reading unit 262 selects the EObuffer 263-1.

If it is determined in step S271 that the type index is not EO (edgeoffset), that is, the type index is band offset, in step S273, thecoefficient reading unit 262 selects the BO buffer 263-2.

In step S274, the coefficient reading unit 262 reads sao_offset (offset)from the position indicated by the copy index (copy_idx) in the selectedbuffer.

For example, as illustrated in FIG. 20, if the type index (sao_type_idx)is 5, the type index indicates band offset. Thus, in step S273, the BObuffer 263-2 is selected. Then, in step S274, sao_offset (offset) isread from the position marked with copy index=3.

As described above, adaptive offset filtering is performed in units ofan LCU, which is the largest coding unit, and the parameters of theadaptive offset filter, which are transmitted in one batch at thebeginning of a frame in the related art, are sequentially sent at thebeginning of each LCU. Thus, whereas a buffer with a capacitycorresponding to one frame is required in the related art, a buffer witha reduced capacity corresponding to an LCU can be used.

In addition, the copy_flag_and the copy index are sent to the decoderside, and a coefficient that has already been used is not sent, thusreducing the increase in the number of sent coefficients, which iscaused by sending a coefficient for each LCU.

Additionally, adaptive offset filtering and adaptive loop filtering canbe performed in units of the same LCU. Accordingly, processingefficiency can be improved.

In addition, a region is split in units of an LCU, thus increasingflexibility in partitioning, compared to that based on a quad-treestructure in the related art. Adaptive offset filtering can be performedin accordance with the characteristics of an input image.

Note that in the foregoing description, the offset buffer 215 and theoffset buffer 263 are constructed in a FIFO, by way of example. However,an FIFO is not given in a limiting sense. That is, the configuration ofthe offset buffer 215 and the offset buffer 263 may be any other bufferconfiguration so long as the same configuration is used on the encoderside and the decoder side.

In the foregoing, the HEVC scheme is basically used as a coding scheme.However, the present disclosure is not limited thereto, and any othercoding scheme/decoding scheme including at least adaptive offsetfiltering as in-loop filtering may be applied.

Note that the present disclosure may be applied to an image encodingdevice and an image decoding device that are used for receiving imageinformation (bit stream) compressed using an orthogonal transform suchas a discrete cosine transform and motion compensation, for example,like the HEVC scheme or the like, via a network medium such as satellitebroadcasting, cable television, the Internet, or a mobile phone. Thepresent disclosure may further be applied to an image encoding deviceand an image decoding device for use in processing on a storage mediumsuch as an optical disk, a magnetic disk, and a flash memory.

4. Second Embodiment Application to Multi-View Image Encoding/Multi-ViewImage Decoding

The series of processes described above can be applied to multi-viewimage encoding and multi-view image decoding. FIG. 21 illustrates anexample of a multi-view image encoding scheme.

As illustrated in FIG. 21, multi-view images include images at aplurality of views, and an image at a certain one of the plurality ofviews is designated as an image of a base view. The images other thanthe image of the base view are handled as images of non-base views.

In a case where multi-view image encoding as in FIG. 21 is performed,adaptive offset filter parameters (a copy flag, a copy index, etc.) canbe set in each view (the same view). In addition, in each view(different views), adaptive offset filter parameters set in another viewcan also be shared.

In this case, adaptive offset filter parameters set in the base view areused in at least one non-base view. Alternatively, for example, adaptiveoffset filter parameters set in a non-base view (view_id=i) are used inat least one of the base view and non-base view (view_id=j).

Accordingly, processing efficiency can be improved.

[Multi-View Image Encoding Device]

FIG. 22 is a diagram illustrating a multi-view image encoding device forperforming the multi-view image encoding operation described above. Asillustrated in FIG. 22, a multi-view image encoding device 600 includesan encoding unit 601, an encoding unit 602, and a multiplexing unit 603.

The encoding unit 601 encodes an image of a base view to generate anencoded base-view image stream. The encoding unit 602 encodes an imageof a non-base view to generate an encoded non-base-view image stream.The multiplexing unit 603 multiplexes the encoded base-view image streamgenerated by the encoding unit 601 and the encoded non-base-view imagestream generated by the encoding unit 602 to generate an encodedmulti-view image stream.

The image encoding device 11 (FIG. 1) can be used for each of theencoding unit 601 and the encoding unit 602 of the multi-view imageencoding device 600. In this case, the multi-view image encoding device600 sets the adaptive offset filter parameters set by the encoding unit601 and the adaptive offset filter parameters set by the encoding unit602, and transmits the set adaptive offset filter parameters.

Note that, as described above, adaptive offset filter parameters set bythe encoding unit 601 may be set so as to be shared and used by theencoding unit 601 and the encoding unit 602, and may be transmitted.Conversely, adaptive offset filter parameters collectively set by theencoding unit 602 may be set so as to be shared and used by the encodingunit 601 and the encoding unit 602, and may be transmitted.

[Multi-View Image Decoding Device]

FIG. 23 is a diagram illustrating a multi-view image decoding device forperforming the multi-view image decoding operation described above. Asillustrated in FIG. 23, a multi-view image decoding device 610 includesa demultiplexing unit 611, a decoding unit 612, and a decoding unit 613.

The demultiplexing unit 611 demultiplexes an encoded multi-view imagestream in which an encoded base-view image stream and an encodednon-base-view image stream have been multiplexed, and extracts theencoded base-view image stream and the encoded non-base-view imagestream. The decoding unit 612 decodes the encoded base-view image streamextracted by the demultiplexing unit 611 to obtain an image of a baseview. The decoding unit 613 decodes the encoded non-base-view imagestream extracted by the demultiplexing unit 611 to obtain an image of anon-base view.

The image decoding device 51 (FIG. 3) can be used for each of thedecoding unit 612 and the decoding unit 613 of the multi-view imagedecoding device 610. In this case, the multi-view image decoding device610 performs processing using the adaptive offset filter parameters setby the encoding unit 601 and decoded by the decoding unit 612 and theadaptive offset filter parameters set by the encoding unit 602 anddecoded by the decoding unit 613.

Note that, as described above, in some cases, adaptive offset filterparameters set by the encoding unit 601 (or the encoding unit 602) maybe set so as to be shared and used by the encoding unit 601 and theencoding unit 602, and may be transmitted. In such cases, in themulti-view image decoding device 610, processing is performed usingadaptive offset filter parameters set by the encoding unit 601 (or theencoding unit 602) and decoded by the decoding unit 612 (or the decodingunit 613).

5. Third Embodiment Application to Layered Image Encoding/Layered ImageDecoding

The series of processes described above may be applied to layered imageencoding and layered image decoding. FIG. 24 illustrates an example of alayered image encoding scheme.

As illustrated in FIG. 24, layered images include images of a pluralityof layers (resolutions), and an image of a certain one of the pluralityof resolutions is designated as an image of a base layer. The imagesother than the image of the base layer are handled as images of non-baselayers (also referred to as enhancement layers).

In a case where layered image encoding (spatial scalability) as in FIG.24 is performed, adaptive offset filter parameters can be set in eachlayer (the same layer). In addition, in each layer (different layers),adaptive offset filter parameters set in another layer can also beshared.

In this case, adaptive offset filter parameters set in the base layerare used in at least one non-base layer. Alternatively, for example,adaptive offset filter parameters set in a non-base layer (layer_id=i)are used in at least one of the base layer and non-base layer(layer_id=j).

Accordingly, processing efficiency can be improved.

[Layered Image Encoding Device]

FIG. 25 is a diagram illustrating a layered image encoding device forperforming the layered image encoding operation described above. Asillustrated in FIG. 25, a layered image encoding device 620 includes anencoding unit 621, an encoding unit 622, and a multiplexing unit 623.

The encoding unit 621 encodes an image of a base layer to generate anencoded base-layer image stream. The encoding unit 622 encodes an imageof a non-base layer to generate an encoded non-base-layer image stream.The multiplexing unit 623 multiplexes the encoded base-layer imagestream generated by the encoding unit 621 and the encoded non-base-layerimage stream generated by the encoding unit 622 to generate an encodedlayered-image stream.

The image encoding device 11 (FIG. 1) can be used for each of theencoding unit 621 and the encoding unit 622 of the layered imageencoding device 620. In this case, the layered image encoding device 620sets the adaptive offset filter parameters set by the encoding unit 621and the adaptive offset filter parameters set by the encoding unit 602,and transmits the set adaptive offset filter parameters.

Note that, as described above, adaptive offset filter parameters set bythe encoding unit 621 may be set so as to be shared and used by theencoding unit 621 and the encoding unit 622, and may be transmitted.Conversely, adaptive offset filter parameters set by the encoding unit622 may be set so as to be shared and used by the encoding unit 621 andthe encoding unit 622, and may be transmitted.

[Layered Image Decoding Device]

FIG. 26 is a diagram illustrating a layered image decoding device forperforming the layered image decoding operation described above. Asillustrated in FIG. 26, a layered image decoding device 630 includes ademultiplexing unit 631, a decoding unit 632, and a decoding unit 633.

The demultiplexing unit 631 demultiplexes an encoded layered-imagestream in which an encoded base-layer image stream and an encodednon-base-layer image stream have been multiplexed, and extracts theencoded base-layer image stream and the encoded non-base-layer imagestream. The decoding unit 632 decodes the encoded base-layer imagestream extracted by the demultiplexing unit 631 to obtain an image of abase layer. The decoding unit 633 decodes the encoded non-base-layerimage stream extracted by the demultiplexing unit 631 to obtain an imageof a non-base layer.

The image decoding device 51 (FIG. 3) can be used for each of thedecoding unit 632 and the decoding unit 633 of the layered imagedecoding device 630. In this case, the layered image decoding device 630performs processing using the adaptive offset filter parameters set bythe encoding unit 621 and decoded by the decoding unit 632 and theadaptive offset filter parameters set by the encoding unit 622 anddecoded by the decoding unit 633.

Note that, as described above, in some cases, adaptive offset filterparameters set by the encoding unit 621 (or the encoding unit 622) maybe set so as to be shared and used by the encoding unit 621 and theencoding unit 622, and may be transmitted. In this case, in the layeredimage decoding device 630, processing is performed using adaptive offsetfilter parameters set by the encoding unit 621 (or the encoding unit622) and decoded by the decoding unit 632 (or the decoding unit 633).

6. Fourth Embodiment Example Configuration of Computer

The series of processes described above may be executed by hardware ormay be executed by software. If the series of processes is to beexecuted by software, a program constituting the software is installedin a computer. Here, examples of the computer include a computerincorporated in dedicated hardware, and a computer capable of executingvarious functions by the installation of various programs, for example,a general-purpose personal computer.

FIG. 27 is a block diagram illustrating an example configuration ofhardware of a computer that executes the series of processes describedabove in accordance with a program.

In a computer 800, a CPU (Central Processing Unit) 801, a ROM (Read OnlyMemory) 802, and a RAM (Random Access Memory) 803 are connected to oneanother via a bus 804.

An input/output interface 805 is further connected to the bus 804. Aninput unit 806, an output unit 807, a storage unit 808, a communicationunit 809, and a drive 810 are connected to the input/output interface805.

The input unit 806 includes a keyboard, a mouse, a microphone, and soforth. The output unit 807 includes a display, a speaker, and so forth.The storage unit 808 includes a hard disk, a non-volatile memory, and soforth. The communication unit 809 includes a network interface and soforth. The drive 810 drives a removable medium 811 such as a magneticdisk, an optical disk, a magneto-optical disk, or a semiconductormemory.

In the computer having the configuration described above, the CPU 801loads a program stored in, for example, the storage unit 808 into theRAM 803 through the input/output interface 805 and the bus 804, andexecutes the program. Accordingly, the series of processes describedabove is performed.

The program executed by the computer 800 (the CPU 801) may be providedin the form of being recorded on the removable medium 811, for example,a package medium. In addition, the program may also be provided througha wired or wireless transmission medium such as a local area network,the Internet, or digital satellite broadcasting.

In the computer, the removable medium 811 is set in the drive 810,thereby allowing the program to be installed into the storage unit 808through the input/output interface 805. In addition, the program may bereceived by the communication unit 809 through a wired or wirelesstransmission medium, and may be installed into the storage unit 808.Alternatively, the program may be installed in advance in the ROM 802 orthe storage unit 808.

Note that the program which the computer executes may be a program inwhich processing operations are performed in a time-series manner in theorder stated herein, or may be a program in which processing operationsare performed in parallel or at necessary timing such as when called.

In addition, steps describing a program stored in a recording medium, asused herein, include, of course, processing operations performed in atime-series manner in the order stated, and processing operationsexecuted in parallel or individually but not necessarily performed in atime-series manner.

Furthermore, the term “system”, as used herein, refers to an overallapparatus including a plurality of devices (apparatuses).

In addition, a configuration described above as a single device (orprocessing unit) may be divided into a plurality of devices (orprocessing units). Conversely, a configuration described above as aplurality of devices (or processing units) may be combined into a singledevice (or processing unit). Additionally, of course, a configurationother than that described above may be added to the configuration ofeach device (or each processing unit). Furthermore, part of theconfiguration of a certain device (or processing unit) may be includedin the configuration of another device (or another processing unit) ifthe devices (or processing units) have substantially the sameconfiguration and/or operation in terms of a whole system. In otherwords, embodiments of the present technology are not limited to theforegoing embodiments, and a variety of modifications can be madewithout departing from the scope of the present technology.

The image encoding device and the image decoding device according to theforegoing embodiments may be applied to various pieces of electronicequipment such as a transmitter or a receiver used to deliver data viasatellite broadcasting, wire broadcasting such as cable TV, or theInternet or used to deliver data to or from terminals via cellularcommunication, a recording apparatus for recording images on media suchas an optical disk, a magnetic disk, and a flash memory, and areproducing apparatus for reproducing images from such storage media.Four exemplary applications will be described hereinafter.

7. Exemplary Applications [First Exemplary Application: TelevisionReceiver]

FIG. 28 illustrates an example of a schematic configuration of atelevision apparatus to which the foregoing embodiments are applied. Atelevision apparatus 900 includes an antenna 901, a tuner 902, ademultiplexer 903, a decoder 904, a video signal processing unit 905, adisplay unit 906, an audio signal processing unit 907, a speaker 908, anexternal interface 909, a control unit 910, a user interface 911, and abus 912.

The tuner 902 extracts a signal on a desired channel from a broadcastsignal received via the antenna 901, and demodulates the extractedsignal. Then, the tuner 902 outputs an encoded bit stream obtainedthrough demodulation to the demultiplexer 903. In other words, the tuner902 functions as a transmission means in the television apparatus 900for receiving an encoded stream including encoded images.

The demultiplexer 903 demultiplexes, from the encoded bit stream, avideo stream and an audio stream of a program to be viewed, and outputsthe demultiplexed streams to the decoder 904. Further, the demultiplexer903 extracts auxiliary data such as EPG (Electronic Program Guide) fromthe encoded bit stream, and supplies the extracted data to the controlunit 910. Note that the demultiplexer 903 may also descramble theencoded bit stream if the encoded bit stream has been scrambled.

The decoder 904 decodes the video stream and audio stream input from thedemultiplexer 903. Then, the decoder 904 outputs video data obtained bya decoding process to the video signal processing unit 905. The decoder904 further outputs audio data generated by a decoding process to theaudio signal processing unit 907.

The video signal processing unit 905 reproduces the video data inputfrom the decoder 904, and causes video to be displayed on the displayunit 906. The video signal processing unit 905 may also cause anapplication screen supplied via a network to be displayed on the displayunit 906. The video signal processing unit 905 may further performadditional processing, such as noise removal, on the video data inaccordance with the settings. In addition, the video signal processingunit 905 may also generate a GUI (Graphical User Interface) image suchas a menu, a button, or a cursor, and superimpose the generated image onan output image.

The display unit 906 is driven by a drive signal supplied from the videosignal processing unit 905, and displays video or an image on a videosurface of a display device (such as a liquid crystal display, a plasmadisplay, or an OELD (Organic ElectroLuminescence Display) (organic ELdisplay)).

The audio signal processing unit 907 performs a reproduction processincluding D/A conversion, amplification, and so forth on the audio datainput from the decoder 904, and causes audio to be output from thespeaker 908. The audio signal processing unit 907 may further performadditional processing, such as noise removal, on the audio data.

The external interface 909 is an interface for connecting the televisionapparatus 900 to an external device or a network. For example, a videostream or audio stream received via the external interface 909 may bedecoded by the decoder 904. In other words, the external interface 909also functions as a transmission means in the television apparatus 900for receiving an encoded stream including encoded images.

The control unit 910 includes a processor such as a CPU, and memoriessuch as a RAM and a ROM. The memories store a program to be executed bythe CPU, program data, EPG data, data acquired via a network, and soforth. The program stored in the memories is read and executed by theCPU when, for example, the television apparatus 900 is started. The CPUexecutes the program to control the operation of the televisionapparatus 900 in accordance with, for example, an operation signal inputfrom the user interface 911.

The user interface 911 is connected to the control unit 910. The userinterface 911 includes, for example, buttons and switches for allowingthe user to operate the television apparatus 900, a reception unit for aremote control signal, and so forth. The user interface 911 detects anoperation of the user via the above-described components to generate anoperation signal, and outputs the generated operation signal to thecontrol unit 910.

The bus 912 serves to connect the tuner 902, the demultiplexer 903, thedecoder 904, the video signal processing unit 905, the audio signalprocessing unit 907, the external interface 909, and the control unit910 to one another.

In the television apparatus 900 having the configuration describedabove, the decoder 904 has the function of the image decoding deviceaccording to the foregoing embodiments. Accordingly, processingefficiency can be improved when the television apparatus 900 decodes animage.

[Second Exemplary Application: Mobile Phone]

FIG. 29 illustrates an example of a schematic configuration of a mobilephone to which the foregoing embodiments are applied. A mobile phone 920includes an antenna 921, a communication unit 922, an audio codec 923, aspeaker 924, a microphone 925, a camera unit 926, an image processingunit 927, a multiplexing/demultiplexing unit 928, arecording/reproducing unit 929, a display unit 930, a control unit 931,an operation unit 932, and a bus 933.

The antenna 921 is connected to the communication unit 922. The speaker924 and the microphone 925 are connected to the audio codec 923. Theoperation unit 932 is connected to the control unit 931. The bus 933serves to connect the communication unit 922, the audio codec 923, thecamera unit 926, the image processing unit 927, themultiplexing/demultiplexing unit 928, the recording/reproducing unit929, the display unit 930, and the control unit 931 to one another.

The mobile phone 920 performs operations, such as transmitting andreceiving an audio signal, transmitting and receiving an electronic mailor image data, capturing an image, and recording data, in variousoperation modes including a voice call mode, a data communication mode,an image capture mode, and a videophone mode.

In the voice call mode, an analog audio signal generated by themicrophone 925 is supplied to the audio codec 923. The audio codec 923converts the analog audio signal into audio data, and performs A/Dconversion and compression on the converted audio data. The audio codec923 then outputs the compressed audio data to the communication unit922. The communication unit 922 encodes and modulates the audio data,and generates a transmission signal. The communication unit 922 thentransmits the generated transmission signal to a base station (notillustrated) via the antenna 921. Further, the communication unit 922amplifies a radio signal received via the antenna 921, and performsfrequency conversion on the amplified signal to acquire a receptionsignal. Then, the communication unit 922 demodulates and decodes thereception signal to generate audio data, and outputs the generated audiodata to the audio codec 923. The audio codec 923 expands the audio data,and performs D/A conversion to generate an analog audio signal. Theaudio codec 923 then supplies the generated audio signal to the speaker924 to cause audio to be output.

Furthermore, in the data communication mode, for example, the controlunit 931 generates text data that forms an electronic mail in accordancewith an operation of the user via the operation unit 932. Further, thecontrol unit 931 causes text to be displayed on the display unit 930.The control unit 931 further generates electronic mail data inaccordance with a transmission instruction given from the user via theoperation unit 932, and outputs the generated electronic mail data tothe communication unit 922. The communication unit 922 encodes andmodulates the electronic mail data to generate a transmission signal.Then, the communication unit 922 transmits the generated transmissionsignal to the base station (not illustrated) via the antenna 921.Further, the communication unit 922 amplifies a radio signal receivedvia the antenna 921, and performs frequency conversion on the amplifiedsignal to acquire a reception signal. Then, the communication unit 922demodulates and decodes the reception signal to restore electronic maildata, and outputs the restored electronic mail data to the control unit931. The control unit 931 causes the content of the electronic mail tobe displayed on the display unit 930, and also causes the electronicmail data to be stored in a storage medium of the recording/reproducingunit 929.

The recording/reproducing unit 929 includes a desired readable/writablestorage medium. The storage medium may be, for example, a built-instorage medium such as a RAM or a flash memory, or an external storagemedium such as a hard disk, a magnetic disk, a magneto-optical disk, anoptical disk, a USB (Unallocated Space Bitmap) memory, or a memory card.

Furthermore, in the image capture mode, for example, the camera unit 926captures an image of an object to generate image data, and outputs thegenerated image data to the image processing unit 927. The imageprocessing unit 927 encodes the image data input from the camera unit926, and causes an encoded stream to be stored in the storage medium ofthe recording/reproducing unit 929.

Furthermore, in the videophone mode, for example, themultiplexing/demultiplexing unit 928 multiplexes the video streamencoded by the image processing unit 927 and the audio stream input fromthe audio codec 923, and outputs a multiplexed stream to thecommunication unit 922. The communication unit 922 encodes and modulatesthe stream to generate a transmission signal. Then, the communicationunit 922 transmits the generated transmission signal to a base station(not illustrated) through the antenna 921. Further, the communicationunit 922 amplifies a radio signal received through the antenna 921, andperforms frequency conversion on the amplified signal to acquire areception signal. The transmission signal and the reception signal mayinclude an encoded bit stream. Then, the communication unit 922demodulates and decodes the reception signal to restore a stream, andoutputs the restored stream to the multiplexing/demultiplexing unit 928.The multiplexing/demultiplexing unit 928 demultiplexes, from the inputstream, a video stream and an audio stream, and outputs the video streamand the audio stream to the image processing unit 927 and the audiocodec 923, respectively. The image processing unit 927 decodes the videostream to generate video data. The video data is supplied to the displayunit 930, and a series of images is displayed by the display unit 930.The audio codec 923 expands the audio stream, and performs D/Aconversion to generate an analog audio signal. The audio codec 923 thensupplies the generated audio signal to the speaker 924 to cause audio tobe output.

In the mobile phone 920 having the configuration described above, theimage processing unit 927 has the function of the image encoding deviceand the image decoding device according to the foregoing embodiments.Accordingly, processing efficiency can be improved when the mobile phone920 encodes and decodes an image.

[Third Exemplary Application: Recording/Reproducing Apparatus]

FIG. 30 illustrates an example of a schematic configuration of arecording/reproducing apparatus to which the foregoing embodiments areapplied. A recording/reproducing apparatus 940 encodes, for example,audio data and video data of a received broadcast program, and recordsthe encoded audio data and video data on a recording medium. Inaddition, the recording/reproducing apparatus 940 may also encode audiodata and video data acquired from, for example, another apparatus, andrecord the encoded audio data and video data on a recording medium.Furthermore, the recording/reproducing apparatus 940 reproduces datarecorded on a recording medium using a monitor and a speaker inaccordance with, for example, an instruction given from a user. In thiscase, the recording/reproducing apparatus 940 decodes audio data andvideo data.

The recording/reproducing apparatus 940 includes a tuner 941, anexternal interface 942, an encoder 943, an HDD (Hard Disk Drive) 944, adisk drive 945, a selector 946, a decoder 947, an OSD (On-ScreenDisplay) 948, a control unit 949, and a user interface 950.

The tuner 941 extracts a signal on a desired channel from a broadcastsignal received via an antenna (not illustrated), and demodulates theextracted signal. The tuner 941 then outputs an encoded bit streamobtained through demodulation to the selector 946. In other words, thetuner 941 functions as a transmission means in the recording/reproducingapparatus 940.

The external interface 942 is an interface for connecting therecording/reproducing apparatus 940 to an external device or a network.The external interface 942 may be, for example, an IEEE 1394 interface,a network interface, a USB interface, a flash memory interface, or thelike. For example, video data and audio data received via the externalinterface 942 are input to the encoder 943. In other words, the externalinterface 942 functions as a transmission means in therecording/reproducing apparatus 940.

The encoder 943 encodes video data and audio data input from theexternal interface 942 if the video data and audio data have not beenencoded. The encoder 943 then outputs an encoded bit stream to theselector 946.

The HDD 944 records an encoded bit stream including compressed contentdata such as video and audio, various programs, and other data on aninternal hard disk. Further, the HDD 944 reads the above-described datafrom the hard disk when reproducing video and audio.

The disk drive 945 records and reads data on and from a recording mediumplaced therein. The recording medium placed in the disk drive 945 maybe, for example, a DVD disk (such as DVD-Video, DVD-RAM, DVD-R, DVD-RW,DVD+R, or DVD+RW), a Blu-ray (registered trademark) disc, or the like.

The selector 946 selects an encoded bit stream input from the tuner 941or the encoder 943 when recording video and audio, and outputs theselected encoded bit stream to the HDD 944 or the disk drive 945. Whenreproducing video and audio, the selector 946 outputs an encoded bitstream input from the HDD 944 or the disk drive 945 to the decoder 947.

The decoder 947 decodes the encoded bit stream to generate video dataand audio data. The decoder 947 then outputs the generated video data tothe OSD 948. The decoder 904 further outputs the generated audio data toan external speaker.

The OSD 948 reproduces the video data input from the decoder 947, anddisplays video. In addition, the OSD 948 may also superimpose a GUIimage such as a menu, a button, or a cursor on the video to bedisplayed.

The control unit 949 includes a processor such as a CPU, and memoriessuch as a RAM and a ROM. The memories store a program to be executed bythe CPU, program data, and so forth. The program stored in the memoriesis read and executed by the CPU when, for example, therecording/reproducing apparatus 940 is started. The CPU executes theprogram to control the operation of the recording/reproducing apparatus940 in accordance with, for example, an operation signal input from theuser interface 950.

The user interface 950 is connected to the control unit 949. The userinterface 950 includes, for example, buttons and switches for allowingthe user to operate the recording/reproducing apparatus 940, a receptionunit for a remote control signal, and so forth. The user interface 950detects an operation of the user via the above-described components togenerate an operation signal, and outputs the generated operation signalto the control unit 949.

In the recording/reproducing apparatus 940 having the configurationdescribed above, the encoder 943 has the function of the image encodingdevice according to the foregoing embodiments. Further, the decoder 947has the function of the image decoding device according to the foregoingembodiments. Accordingly, processing efficiency can be improved when therecording/reproducing apparatus 940 encodes and decodes an image.

[Fourth Exemplary Application: Imaging Apparatus]

FIG. 31 illustrates an example of a schematic configuration of animaging apparatus to which the foregoing embodiments are applied. Animaging apparatus 960 captures an image of an object to generate imagedata, encodes the image data, and records the encoded image data on arecording medium.

The imaging apparatus 960 includes an optical block 961, an imaging unit962, a signal processing unit 963, an image processing unit 964, adisplay unit 965, an external interface 966, a memory 967, a mediumdrive 968, an OSD 969, a control unit 970, a user interface 971, and abus 972.

The optical block 961 is connected to the imaging unit 962. The imagingunit 962 is connected to the signal processing unit 963. The displayunit 965 is connected to the image processing unit 964. The userinterface 971 is connected to the control unit 970. The bus 972 servesto connect the image processing unit 964, the external interface 966,the memory 967, the medium drive 968, the OSD 969, and the control unit970 to one another.

The optical block 961 includes a focus lens, an aperture mechanism, andso forth. The optical block 961 forms an optical image of the object onan imaging surface of the imaging unit 962. The imaging unit 962includes an image sensor such as a CCD (Charge Coupled Device) or CMOS(Complementary Metal Oxide Semiconductor) image sensor, and converts theoptical image formed on the imaging surface into an image signal servingas an electrical signal by performing photoelectric conversion. Theimaging unit 962 then outputs the image signal to the signal processingunit 963.

The signal processing unit 963 performs various camera signal processingoperations, such as knee correction, gamma correction, and colorcorrection, on the image signal input from the imaging unit 962. Thesignal processing unit 963 outputs the image data subjected to thecamera signal processing operations to the image processing unit 964.

The image processing unit 964 encodes the image data input from thesignal processing unit 963 to generate encoded data. The imageprocessing unit 964 then outputs the generated encoded data to theexternal interface 966 or the medium drive 968. Further, the imageprocessing unit 964 decodes the encoded data input from the externalinterface 966 or the medium drive 968 to generate image data. The imageprocessing unit 964 then outputs the generated image data to the displayunit 965. In addition, the image processing unit 964 may also output theimage data input from the signal processing unit 963 to the display unit965 to cause an image to be displayed. In addition, the image processingunit 964 may also superimpose display data acquired from the OSD 969 onthe image to be output to the display unit 965.

The OSD 969 generates a GUI image such as a menu, a button, or a cursor,and outputs the generated image to the image processing unit 964.

The external interface 966 is formed as, for example, a USB input/outputterminal. The external interface 966 connects, for example, the imagingapparatus 960 to a printer when printing an image. A drive is furtherconnected to the external interface 966, if necessary. A removablemedium such as a magnetic disk or an optical disk is placed in thedrive, and a program read from the removable medium may be installedinto the imaging apparatus 960. In addition, the external interface 966may also be formed as a network interface to be connected to a networksuch as a LAN or the Internet. In other words, the external interface966 functions as a transmission means in the imaging apparatus 960.

The recording medium to be placed in the medium drive 968 may be, forexample, any readable/writable removable medium such as a magnetic disk,a magneto-optical disk, an optical disk, or a semiconductor memory.Alternatively, a recording medium may be fixedly attached to the mediumdrive 968, and may form a built-in hard disk drive or a non-portablestorage unit such as an SSD (Solid State Drive).

The control unit 970 includes a processor such as a CPU, and memoriessuch as a RAM and a ROM. The memories store a program to be executed bythe CPU, program data, and so forth. The program stored in the memoriesis read and executed by the CPU when, for example, the imaging apparatus960 is started. The CPU executes the program to control the operation ofthe imaging apparatus 960 in accordance with, for example, an operationsignal input from the user interface 971.

The user interface 971 is connected to the control unit 970. The userinterface 971 includes, for example, buttons, switches, and so forth forallowing the user to operate the imaging apparatus 960. The userinterface 971 detects an operation of the user via the above-describedcomponents to generate an operation signal, and outputs the generatedoperation signal to the control unit 970.

In the imaging apparatus 960 having the configuration described above,the image processing unit 964 has the function of the image encodingdevice and the image decoding device according to the foregoingembodiments. Accordingly, processing efficiency can be improved when theimaging apparatus 960 encodes and decodes an image.

8. Exemplary Applications of Scalable Coding

[First System]

Next, a specific example of use of scalable coded data which has beenscalably coded (hierarchically coded) described above with reference toFIG. 24 to FIG. 26 will be described. Scalable coding is used for, forexample, the selection of data to be transmitted, as in an exampleillustrated in FIG. 32.

In a data transmission system 1000 illustrated in FIG. 32, adistribution server 1002 reads scalable coded data stored in a scalablecoded data storage unit 1001, and distributes the scalable coded data toterminal devices, such as a personal computer 1004, an AV device 1005, atablet device 1006, and a mobile phone 1007, via a network 1003.

In this case, the distribution server 1002 selects encoded data havingdesired quality in accordance with the performance of the terminaldevice, the communication environment, and the like, and transmits theselected encoded data. Even if the distribution server 1002 transmitsdata having quality higher than necessary, the terminal device may notbe able to always obtain a high-quality image, and delay or overflow maybe caused. In addition, such data may occupy the communication bandwidthmore than necessary, or may increase the load on the terminal devicemore than necessary. Conversely, even if the distribution server 1002transmits data having quality lower than necessary, the terminal devicemay not be able to obtain an image with a sufficient quality. Thus, thedistribution server 1002 reads the scalable coded data stored in thescalable coded data storage unit 1001, if necessary, as encoded datahaving quality appropriate for the performance of the terminal device,communication environment, and the like, and transmits the read encodeddata.

For example, it is assumed that the scalable coded data storage unit1001 stores scalable coded data (BL+EL) 1011 which has been scalablycoded. The scalable coded data (BL+EL) 1011 is encoded data including abase layer and an enhancement layer, and is data which is decoded toobtain both an image of the base layer and an image of the enhancementlayer.

The distribution server 1002 selects an appropriate layer in accordancewith the performance of a terminal device that transmits data, thecommunication environment, and the like, and reads the data of thelayer. For example, the distribution server 1002 reads high-qualityscalable coded data (BL+EL) 1011 from the scalable coded data storageunit 1001, and transmits the read scalable coded data (BL+EL) 1011 todevices having high processing capabilities, namely, the personalcomputer 1004 or the tablet device 1006, as it is. In contrast, forexample, the distribution server 1002 extracts the data of the baselayer from the scalable coded data (BL+EL) 1011, and transmits theextracted data of the base layer to devices having low processingcapabilities, namely, the AV device 1005 and the mobile phone 1007, asscalable coded data (BL) 1012 having the same content as the scalablecoded data (BL+EL) 1011 but having lower quality than the scalable codeddata (BL+EL) 1011.

The use of scalable coded data in this manner facilitates the adjustmentof the amount of data, thereby suppressing the occurrence of delay oroverflow and suppressing an unnecessary increase in the load on aterminal device or a communication medium. Furthermore, the scalablecoded data (BL+EL) 1011 has reduced redundancy between layers, andtherefore has a smaller amount of data than data having individuallyencoded data of the respective layers. Accordingly, the storage area ofthe scalable coded data storage unit 1001 can be more efficientlyutilized.

Note that since various devices such as the personal computer 1004, theAV device 1005, the tablet device 1006, and the mobile phone 1007 areapplicable as terminal devices, the hardware performance of terminaldevices differs from device to device. In addition, since variousapplications may be executed by terminal devices, the softwarecapabilities of the applications may vary. Furthermore, the network 1003serving as a communication medium may be implemented as anycommunication line network which can be wired, wireless, or both, suchas the Internet and a LAN (Local Area Network), and data transmissioncapabilities vary. Such performance and capabilities may vary dependingon other communication and the like.

Accordingly, prior to the start of transmission of data, thedistribution server 1002 may communicate with a terminal device to whichthe data is to be transmitted, and may obtain information concerning thecapabilities of the terminal device, such as the hardware performance ofthe terminal device or the performance of application (software)executed by the terminal device, and also information concerning thecommunication environment, such as the available bandwidth of thenetwork 1003. In addition, the distribution server 1002 may select anappropriate layer on the basis of the obtained information.

Note that a layer may be extracted by a terminal device. For example,the personal computer 1004 may decode the transmitted scalable codeddata (BL+EL) 1011, and display an image of a base layer or an image ofan enhancement layer. Alternatively, for example, the personal computer1004 may extract the scalable coded data (BL) 1012 of the base layerfrom the transmitted scalable coded data (BL+EL) 1011, store theextracted scalable coded data (BL) 1012, transfer the extracted scalablecoded data (BL) 1012 to another device, or decode the extracted scalablecoded data (BL) 1012 to display an image of the base layer.

As a matter of course, the number of scalable coded data storage units1001, the number of distribution servers 1002, the number of networks1003, and the number of terminal devices may be arbitrary. In addition,while a description has been given of an example in which thedistribution server 1002 transmits data to a terminal device, examplesof use are not limited thereto. The data transmission system 1000 may beused in any system that selects and transmits an appropriate layer, whentransmitting encoded data which has been scalably coded to a terminaldevice, in accordance with the capabilities of the terminal device, thecommunication environment, and the like.

In addition, the present technology can also be applied to the datatransmission system 1000 illustrated in FIG. 32 as described above in amanner similar to application to the hierarchical encoding andhierarchical decoding described above with reference to FIG. 24 to FIG.26, thereby achieving advantages similar to the advantages describedabove with reference to FIG. 24 to FIG. 26.

[Second System]

Scalable coding is also used for, for example, as in an exampleillustrated in FIG. 33, transmission via a plurality of communicationmedia.

In a data transmission system 1100 illustrated in FIG. 33, a broadcaststation 1101 transmits scalable coded data (BL) 1121 of a base layer viaterrestrial broadcasting 1111. The broadcast station 1101 furthertransmits (e.g. packetizes and transmits) scalable coded data (EL) 1122of an enhancement layer via a desired network 1112 formed of acommunication network which can be wired, wireless, or both.

A terminal device 1102 has a function for receiving the terrestrialbroadcasting 1111 from the broadcast station 1101, and receives thescalable coded data (BL) 1121 of the base layer transmitted via theterrestrial broadcasting 1111. The terminal device 1102 further has acommunication function for performing communication via the network1112, and receives the scalable coded data (EL) 1122 of the enhancementlayer transmitted via the network 1112.

The terminal device 1102 decodes the scalable coded data (BL) 1121 ofthe base layer acquired via the terrestrial broadcasting 1111 inaccordance with, for example, a user instruction or the like to obtainan image of the base layer, stores the scalable coded data (BL) 1121, ortransfers the scalable coded data (BL) 1121 to another device.

Further, the terminal device 1102 combines the scalable coded data (BL)1121 of the base layer acquired via the terrestrial broadcasting 1111with the scalable coded data (EL) 1122 of the enhancement layer acquiredvia the network 1112 in accordance with, for example, a user instructionor the like to obtain scalable coded data (BL+EL), and decodes thescalable coded data (BL+EL) to obtain an image of the enhancement layer,stores the scalable coded data (BL+EL), or transfers the scalable codeddata (BL+EL) to another device.

As described above, scalable coded data can be transmitted via, forexample, a communication medium that is different from one layer toanother. Thus, the load can be distributed, and delay or overflow can besuppressed from occurring.

Further, a communication medium to be used for transmission may be madeselectable for each layer in accordance with the situation. For example,the scalable coded data (BL) 1121 of the base layer having a relativelylarge amount of data may be transmitted via a communication mediumhaving a large bandwidth, and the scalable coded data (EL) 1122 of theenhancement layer having a relatively small amount of data may betransmitted via a communication medium having a narrow bandwidth.Alternatively, for example, the communication medium via which thescalable coded data (EL) 1122 of the enhancement layer is to betransmitted may be switched between the network 1112 and the terrestrialbroadcasting 1111 in accordance with the available bandwidth of thenetwork 1112. As a matter of course, the above similarly applies to dataof an arbitrary layer.

Control in the manner described above can further suppress an increasein the load of data transmission.

As a matter of course, the number of layers is arbitrary, and the numberof communication media to be used for transmission is also arbitrary. Inaddition, the number of terminal devices 1102 to which data is to bedistributed is also arbitrary. In addition, while a description has beengiven in the context of broadcasting from the broadcast station 1101 byway of example, examples of use are not limited thereto. The datatransmission system 1100 may be applied to any system that dividesencoded data which has been subjected to scalable coding into aplurality of segments in units of layers and that transmits the datasegments via a plurality of lines.

Furthermore, the present technology can also be applied to the datatransmission system 1100 illustrated in FIG. 33 as described above in amanner similar to application to the hierarchical encoding andhierarchical decoding described above with reference to FIG. 24 to FIG.26, thereby achieving advantages similar to the advantages describedabove with reference to FIG. 24 to FIG. 26.

[Third System]

Scalable coding is also used for, for example, as in an exampleillustrated in FIG. 34, the storage of encoded data.

In an imaging system 1200 illustrated in FIG. 34, an imaging apparatus1201 performs scalable coding on image data obtained by capturing animage of an object 1211, and supplies the resulting data to a scalablecoded data storage device 1202 as scalable coded data (BL+EL) 1221.

The scalable coded data storage device 1202 stores the scalable codeddata (BL+EL) 1221 supplied from the imaging apparatus 1201 at thequality corresponding to the situation. For example, in normal time, thescalable coded data storage device 1202 extracts data of a base layerfrom the scalable coded data (BL+EL) 1221, and stores the extracted dataof the base layer as scalable coded data (BL) 1222 of the base layerhaving a low quality and a small amount of data. In contrast, forexample, in attention time, the scalable coded data storage device 1202stores the scalable coded data (BL+EL) 1221 having a high quality and alarge amount of data, as it is.

Accordingly, the scalable coded data storage device 1202 can save animage at high quality only when necessary. This can suppress an increasein the amount of data while suppressing a reduction in the worth of theimage due to a reduction in quality, and can improve use efficiency ofthe storage area.

For example, it is assumed that the imaging apparatus 1201 is a securitycamera. If an object to be monitored (e.g., intruder) does not appear ina captured image (normal time), it may be probable that the capturedimage does not have important content. Thus, a reduction in the amountof data is prioritized, and the image data (scalable coded data) of theimage is stored at low quality. In contrast, if an object to bemonitored appears as the object 1211 in a captured image (attentiontime), it may be probable that the captured image has important content.Thus, image quality is prioritized, and the image data (scalable codeddata) of the image is stored at high quality.

Note that either the normal time or the attention time may be determinedby, for example, the scalable coded data storage device 1202 byanalyzing an image. Alternatively, the imaging apparatus 1201 maydetermine the normal time or the attention time, and may transmit thedetermination result to the scalable coded data storage device 1202.

Note that the determination of either the normal time or the attentiontime may be based on an arbitrary standard, and an image on which thedetermination is based may have any content. As a matter of course,conditions other than the content of an image may be used as thedetermination standard. The state may be changed in accordance with, forexample, the magnitude, waveform, or the like of recorded audio, or maybe changed at intervals of a predetermined period of time.Alternatively, the state may be changed in accordance with an externalinstruction such as a user instruction.

In addition, while a description has been given of an example ofchanging between two states, namely, normal time and attention time, thenumber of states is arbitrary, and the state change may be made betweenmore than two states, such as normal time, less attention time,attention time, and highly attention time. Note that the upper limitnumber of states to be changed depends on the number of layers ofscalable coded data.

Furthermore, the imaging apparatus 1201 may be configured to determinethe number of layers of scalable coding in accordance with the state.For example, in normal time, the imaging apparatus 1201 may generatescalable coded data (BL) 1222 of the base layer having a low quality anda small amount of data, and supply the generated scalable coded data(BL) 1222 to the scalable coded data storage device 1202. Further, forexample, in attention time, the imaging apparatus 1201 may generatescalable coded data (BL+EL) 1221 of the base layer having a high qualityand a large amount of data, and supply the generated scalable coded data(BL+EL) 1221 to the scalable coded data storage device 1202.

While a security camera has been described as an example, the imagingsystem 1200 may be used in any application, and the application is notlimited to a security camera.

In addition, the present technology can also be applied to the imagingsystem 1200 illustrated in FIG. 34 as described above in a mannersimilar to application to the hierarchical encoding and hierarchicaldecoding described above with reference to FIG. 24 to FIG. 26, therebyachieving advantages similar to the advantages described above withreference to FIG. 24 to FIG. 26.

Note that an example in which various pieces of information such as theparameters of the adaptive offset filter are multiplexed in an encodedstream and are transmitted from the encoder side to the decoder side hasbeen described herein. However, the technique for transmitting suchpieces of information is not limited to this example. For example, thesepieces of information may be transmitted or recorded as separate dataassociated with the encoded bit stream without being multiplexed intothe encoded bit stream. The term “associate”, as used herein, meansallowing an image (which may be part of an image, such as a slice orblock) included in a bit stream to be linked to informationcorresponding to the image when the image is decoded. That is, theinformation may be transmitted on a transmission path different fromthat for the image (or bit stream). Further, the information may berecorded on a recording medium different from that for the image (or bitstream) (or recorded in a different recording area of the same recordingmedium). Furthermore, the information and the image (or bit stream) maybe associated with each other in arbitrary units such as a plurality offrames, one frame, or a portion in a frame.

While preferred embodiments of the present disclosure have beendescribed in detail with reference to the accompanying drawings, thepresent disclosure is not limited to such examples. It is apparent thatany person having ordinary knowledge in the field of art to which thepresent disclosure pertains could achieve various changes ormodifications within the scope of the technical idea as defined in theCLAIMS, and it is to be understood that such changes or modificationsmay also fall within the technical scope of the present disclosure.

Note that the present technology may also provide followingconfigurations.

(1) An image processing device including:

an acquisition unit that acquires parameters of an adaptive offsetfilter in units of a largest coding unit from an encoded stream in whichthe parameters of the adaptive offset filter are set using the largestcoding unit as a unit of transmission;

a decoding unit that performs a decoding process on the encoded streamand that generates an image; and

an adaptive offset filter unit that performs adaptive offset filteringon the image generated by the decoding unit, in units of a largestcoding unit using the parameters acquired by the acquisition unit.

(2) The image processing device according to (1) above, wherein

the parameters of the adaptive offset filter include a type of theadaptive offset filter and an offset value.

(3) The image processing device according to (1) or (2) above, furthercomprising

a deblocking filter unit that performs deblocking filtering on the imagegenerated by the decoding unit,

wherein the adaptive offset filter unit performs adaptive offsetfiltering on an image on which the deblocking filter unit has performeddeblocking filtering.

(4) The image processing device according to any of (1) through (3)above, wherein

the acquisition unit acquires, from the encoded stream, identificationdata identifying a parameter of a previous largest coding unit on whichadaptive offset filtering has been performed before a current largestcoding unit as being the same as a parameter of the current largestcoding unit, and

the adaptive offset filter unit performs adaptive offset filtering onthe image generated by the decoding unit, in units of a largest codingunit using the identification data acquired by the acquisition unit.

(5) The image processing device according to any of (1) through (3)above, wherein

the acquisition unit acquires, from the encoded stream, identificationdata identifying whether or not to use a parameter of a previous largestcoding unit on which adaptive offset filtering has been performed beforea current largest coding unit, and

the adaptive offset filter unit performs adaptive offset filtering onthe image generated by the decoding unit, in units of a largest codingunit using the identification data acquired by the acquisition unit.

(6) The image processing device according to any of (1) through (3)above, wherein

the acquisition unit acquires, from the encoded stream, identificationdata identifying whether or not to use a copy of a parameter of aprevious largest coding unit on which adaptive offset filtering has beenperformed before a current largest coding unit, and

the adaptive offset filter unit performs adaptive offset filtering onthe image generated by the decoding unit, in units of a largest codingunit using the identification data acquired by the acquisition unit.

(7) The image processing device according to any of (1) through (3)above, wherein

the acquisition unit acquires, from the encoded stream, identificationdata specifying a largest coding unit that is identical to a currentlargest coding unit in terms of a parameter, from within previouslargest coding units on which adaptive offset filtering has beenperformed before the current largest coding unit, and

the adaptive offset filter unit performs adaptive offset filtering onthe image generated by the decoding unit, in units of a largest codingunit using the identification data acquired by the acquisition unit.

(8) The image processing device according to any of (1) through (7)above, wherein

the parameters of the adaptive offset filter are transmitted at timingof the beginning of a largest coding unit.

(9) The image processing device according to any of (1) through (8)above, wherein

the decoding unit performs a decoding process in units each having ahierarchical structure.

(10) An image processing method including:

acquiring parameters of an adaptive offset filter in units of a largestcoding unit from an encoded stream in which the parameters of theadaptive offset filter are set using the largest coding unit as a unitof transmission;

performing a decoding process on the encoded stream to generate animage; and

performing adaptive offset filtering on the generated image in units ofa largest coding unit using the acquired parameters,

wherein the image processing method is performed by an image processingdevice.

(11) An image processing device including:

a setting unit that sets parameters of an adaptive offset filter using alargest coding unit as a unit of transmission;

an adaptive offset filter unit that performs adaptive offset filteringon an image which has been subjected to a local decoding process in acase where an image is encoded, in units of a largest coding unit usingthe parameters set by the setting unit;

an encoding unit that performs an encoding process on an image on whichthe adaptive offset filter unit has performed adaptive offset filtering,and that generates an encoded stream using the image; and

a transmission unit that transmits the parameters set by the settingunit and the encoded stream generated by the encoding unit.

(12) The image processing device according to (11) above, wherein

the parameters of the adaptive offset filter include a type of theadaptive offset filter and an offset value.

(13) The image processing device according to (11) or (12) above,further including

a deblocking filter unit that performs deblocking filtering on a locallydecoded image,

wherein the adaptive offset filter unit performs adaptive offsetfiltering on an image on which the deblocking filter unit has performeddeblocking filtering.

(14) The image processing device according to any of (11) through (13)above, wherein

the setting unit sets identification data identifying a parameter of aprevious largest coding unit on which adaptive offset filtering has beenperformed before a current largest coding unit as being the same as aparameter of the current largest coding unit, and

the transmission unit transmits the identification data set by thesetting unit and the encoded stream generated by the encoding unit.

(15) The image processing device according to any of (11) through (13)above, wherein

the setting unit sets identification data identifying whether or not touse a parameter of a previous largest coding unit on which adaptiveoffset filtering has been performed before a current largest codingunit, and the transmission unit transmits the identification data set bythe setting unit and the encoded stream generated by the encoding unit.

(16) The image processing device according to any of (11) through (13)above, wherein

the setting unit sets identification data identifying whether or not touse a copy of a parameter of a previous largest coding unit on whichadaptive offset filtering has been performed before a current largestcoding unit, and

the transmission unit transmits the identification data set by thesetting unit and the encoded stream generated by the encoding unit.

(17) The image processing device according to any of (11) through (13)above, wherein

the setting unit sets identification data specifying a largest codingunit that is identical to a current largest coding unit in terms of aparameter, from within previous largest coding units on which adaptiveoffset filtering has been performed before the current largest codingunit, and

the transmission unit transmits the identification data set by thesetting unit and the encoded stream generated by the encoding unit.

(18) The image processing device according to any of (11) through (17)above, wherein

the transmission unit transmits the parameters of the adaptive offsetfilter set by the setting unit at timing of the beginning of a largestcoding unit.

(19) The image processing device according to any of (11) through (18)above, wherein

the encoding unit performs an encoding process in units each having ahierarchical structure.

(20) An image processing method including:

setting parameters of an adaptive offset filter using a largest codingunit as a unit of transmission;

performing adaptive offset filtering on an image which has beensubjected to a local decoding process in a case where an image isencoded, in units of a largest coding unit using the set parameters;

performing an encoding process on an image on which adaptive offsetfiltering has been performed, to generate an encoded stream using theimage; and

transmitting the set parameters and the generated encoded stream,

wherein the image processing method is performed by an image processingdevice.

REFERENCE SIGNS LIST

11 image encoding device, 26 lossless encoding unit, deblocking filter,41 adaptive offset filter, 42 adaptive loop filter, 51 image decodingdevice, 62 lossless decoding unit, 66 deblocking filter, 81 adaptiveoffset filter, 82 adaptive loop filter, 211 type and offsetdetermination unit, 212 offset processing unit, 213 image buffer, 214coefficient reading unit, 215 offset buffer, 215-1 BO buffer, 215-2 BObuffer, 216 parameter setting unit, 217 coefficient writing unit, 221syntax writing unit, 251 syntax reading unit, 261 parameter receivingunit, 262 coefficient reading unit, 263 offset buffer, 263-1 EO buffer,263-2 BO buffer, 264 coefficient writing unit, 265 offset processingunit, 266 image buffer

1. An image processing device comprising: an acquisition unit thatacquires parameters of an adaptive offset filter in units of a largestcoding unit from an encoded stream in which the parameters of theadaptive offset filter are set using the largest coding unit as a unitof transmission; a decoding unit that performs a decoding process on theencoded stream and that generates an image; and an adaptive offsetfilter unit that performs adaptive offset filtering on the imagegenerated by the decoding unit, in units of a largest coding unit usingthe parameters acquired by the acquisition unit.
 2. The image processingdevice according to claim 1, wherein the parameters of the adaptiveoffset filter include a type of the adaptive offset filter and an offsetvalue.
 3. The image processing device according to claim 1, furthercomprising a deblocking filter unit that performs deblocking filteringon the image generated by the decoding unit, wherein the adaptive offsetfilter unit performs adaptive offset filtering on an image on which thedeblocking filter unit has performed deblocking filtering.
 4. The imageprocessing device according to claim 1, wherein the acquisition unitacquires, from the encoded stream, identification data identifying aparameter of a previous largest coding unit on which adaptive offsetfiltering has been performed before a current largest coding unit asbeing the same as a parameter of the current largest coding unit, andthe adaptive offset filter unit performs adaptive offset filtering onthe image generated by the decoding unit, in units of a largest codingunit using the identification data acquired by the acquisition unit. 5.The image processing device according to claim 1, wherein theacquisition unit acquires, from the encoded stream, identification dataidentifying whether or not to use a parameter of a previous largestcoding unit on which adaptive offset filtering has been performed beforea current largest coding unit, and the adaptive offset filter unitperforms adaptive offset filtering on the image generated by thedecoding unit, in units of a largest coding unit using theidentification data acquired by the acquisition unit.
 6. The imageprocessing device according to claim 1, wherein the acquisition unitacquires, from the encoded stream, identification data identifyingwhether or not to use a copy of a parameter of a previous largest codingunit on which adaptive offset filtering has been performed before acurrent largest coding unit, and the adaptive offset filter unitperforms adaptive offset filtering on the image generated by thedecoding unit, in units of a largest coding unit using theidentification data acquired by the acquisition unit.
 7. The imageprocessing device according to claim 1, wherein the acquisition unitacquires, from the encoded stream, identification data specifying alargest coding unit that is identical to a current largest coding unitin terms of a parameter, from within previous largest coding units onwhich adaptive offset filtering has been performed before the currentlargest coding unit, and the adaptive offset filter unit performsadaptive offset filtering on the image generated by the decoding unit,in units of a largest coding unit using the identification data acquiredby the acquisition unit.
 8. The image processing device according toclaim 1, wherein the parameters of the adaptive offset filter aretransmitted at timing of the beginning of a largest coding unit.
 9. Theimage processing device according to claim 1, wherein the decoding unitperforms a decoding process in units each having a hierarchicalstructure.
 10. An image processing method comprising: acquiringparameters of an adaptive offset filter in units of a largest codingunit from an encoded stream in which the parameters of the adaptiveoffset filter are set using the largest coding unit as a unit oftransmission; performing a decoding process on the encoded stream togenerate an image; and performing adaptive offset filtering on thegenerated image in units of a largest coding unit using the acquiredparameters, wherein the image processing method is performed by an imageprocessing device.
 11. An image processing device comprising: a settingunit that sets parameters of an adaptive offset filter using a largestcoding unit as a unit of transmission; an adaptive offset filter unitthat performs adaptive offset filtering on an image which has beensubjected to a local decoding process in a case where an image isencoded, in units of a largest coding unit using the parameters set bythe setting unit; an encoding unit that performs an encoding process onan image on which the adaptive offset filter unit has performed adaptiveoffset filtering, and that generates an encoded stream using the image;and a transmission unit that transmits the parameters set by the settingunit and the encoded stream generated by the encoding unit.
 12. Theimage processing device according to claim 11, wherein the parameters ofthe adaptive offset filter include a type of the adaptive offset filterand an offset value.
 13. The image processing device according to claim11, further comprising a deblocking filter unit that performs deblockingfiltering on a locally decoded image, wherein the adaptive offset filterunit performs adaptive offset filtering on an image on which thedeblocking filter unit has performed deblocking filtering.
 14. The imageprocessing device according to claim 11, wherein the setting unit setsidentification data identifying a parameter of a previous largest codingunit on which adaptive offset filtering has been performed before acurrent largest coding unit as being the same as a parameter of thecurrent largest coding unit, and the transmission unit transmits theidentification data set by the setting unit and the encoded streamgenerated by the encoding unit.
 15. The image processing deviceaccording to claim 11, wherein the setting unit sets identification dataidentifying whether or not to use a parameter of a previous largestcoding unit on which adaptive offset filtering has been performed beforea current largest coding unit, and the transmission unit transmits theidentification data set by the setting unit and the encoded streamgenerated by the encoding unit.
 16. The image processing deviceaccording to claim 11, wherein the setting unit sets identification dataidentifying whether or not to use a copy of a parameter of a previouslargest coding unit on which adaptive offset filtering has beenperformed before a current largest coding unit, and the transmissionunit transmits the identification data set by the setting unit and theencoded stream generated by the encoding unit.
 17. The image processingdevice according to claim 11, wherein the setting unit setsidentification data specifying a largest coding unit that is identicalto a current largest coding unit in terms of a parameter, from withinprevious largest coding units on which adaptive offset filtering hasbeen performed before the current largest coding unit, and thetransmission unit transmits the identification data set by the settingunit and the encoded stream generated by the encoding unit.
 18. Theimage processing device according to claim 11, wherein the transmissionunit transmits the parameters of the adaptive offset filter set by thesetting unit at timing of the beginning of a largest coding unit. 19.The image processing device according to claim 11, wherein the encodingunit performs an encoding process in units each having a hierarchicalstructure.
 20. An image processing method comprising: setting parametersof an adaptive offset filter using a largest coding unit as a unit oftransmission; performing adaptive offset filtering on an image which hasbeen subjected to a local decoding process in a case where an image isencoded, in units of a largest coding unit using the set parameters;performing an encoding process on an image on which adaptive offsetfiltering has been performed, to generate an encoded stream using theimage; and transmitting the set parameters and the generated encodedstream, wherein the image processing method is performed by an imageprocessing device.